# Automatically counting the number of lines of code in a set of notebooks

How can I automatically count the number of lines of code "Input" existing inside all the notebook files contained in a specified folder?

I know there is the issue of considering or not the automatic line breakings as new lines. But considering the purpose of this exercise (statistically convince that the amount of developed work on Mathematica justifies keeping a license), I'm not searching for exact values.

-
If this is a metric for the volume of code, perhaps LeafCount is better for Mathematica. What do you think? Mathematica doesn't respect line breaks and tends to put everything on the same line when converting between e.g. InputForm and StandardForm. –  Szabolcs Feb 13 '12 at 18:32
Given the goal,wouldn't a page count do as well? In that case you could use Heike's code here. Or convert the page count to a line count by finding out how many lines fit on a page. –  Sjoerd C. de Vries Feb 13 '12 at 19:00
Sorry for taking so long to come back... This is to compare with transforming all code to MATLAB, or a standard .NET code. This means that it has to be apples with apples; and I think the most typical are the kloc (lines of code) –  P. Fonseca Feb 13 '12 at 20:07

Assuming that the notebook object is contained in the variable nb (for the current notebook do nb = EvaluationNotebook[]), one could use the following:

StringCount[
First@FrontEndExecute@FrontEnd`ExportPacket[
Notebook@Cases[First@NotebookGet[nb], Cell[_, "Input"|"Code", ___]],
"PlainText"],
"\n"
]

Using "InputText" will wrap lines much more frequently and will give a substantially larger count.

This is equivalent to Alt-clicking an input cell to select all input cells, then copying the contents as "plain text" or "input text" and counting the lines.

As you can see, Mathematica does not really respect line breaks. If you are trying to measure the volume of code, I suggest using the LeafCount of expressions instead of counting code lines.

-
@Leonid This is the same as copying the code using the context menu, so it will never contain any \[IndentingNewLine]. Did you find what is causing the errors in those notebooks? –  Szabolcs Feb 13 '12 at 19:16
Ok, that makes sense - probably I was wrong about new lines then. Looks like those errors are caused by some escaped new lines. I am getting errors like this: "String or list of strings expected at position [NoBreak]1[NoBreak] \in [NoBreak]StringCount[First[$Failed],\!(\"\\\\n\")]" – Leonid Shifrin Feb 13 '12 at 19:48 Regarding line breaks: I think I know what you meant, but literally speaking, Mathematica does respect linebreaks. In oher words, at any given time you may check what the box expression is, and every time you see there "\n" or "[IndentingNewLine]", you can be sure there will be a linebreak. Sometimes, Mathematica breaks excessively long lines without inserting linebreaks, just as other editors do - but those should not be considered linebreaks anyway, I think. – Leonid Shifrin Feb 13 '12 at 20:02 @Leonid What I meant was that when you paste code around (paste it to a text editor, then back, or paste from a website to Mathematica), line breaks get shuffled around. Also converting a cell to e.g. StandardForm (ctrl-shift-n) will eat linebreaks. – Szabolcs Feb 13 '12 at 20:10 Ok, I see - just didn't get what you meant. There seems to be a real impedance mismatch between the FE interface (notebooks) and code editing /coding, as we are used to, in other languages, and this is just one facet of it. I think, often it negatively affects productivity, except in the earliest exploratory stage. I am actually inclined to move to "Program"-style cells also for Mathematica code, probably will do this some time soon. – Leonid Shifrin Feb 13 '12 at 20:20 Here is my version: ClearAll[nbLines]; nbLines[nb_NotebookObject] := With[{inputs = Cases[NotebookGet[nb], Cell[BoxData[data_], "Input", ___] :> data, Infinity] }, Length[inputs] + Count[Characters@StringJoin@Cases[inputs, _String, Infinity], "\n" | "\[IndentingNewLine]"] ] This may not be precise (because the method of input extraction is not totally robust), but should be good enough for qualitative estimate. - If the notebook contains only input the following rough approach using Import will do: StringCases[Import["ExampleData/document.nb", "Plaintext"] , Longest[ {"\n" | "\r"} ..]] // Length (* ==> 19 *) If there are output cells with lines of text (graphics output was already removed by the Plaintext import), we can remove the output cells first, export to string as NB and re-import as plain text counting lines of input: StringCases[ ImportString[ ExportString[ Import["ExampleData/document.nb"] /. Cell[___, "Output", ___] -> Sequence[], "NB" ], "Plaintext" ], Longest[ {"\n" | "\r"} ..] ] // Length (* ==> 15 *) - Somehow I do like kernel-only solutions: (* see also the github repository here *) CountNumberOfLinesOfInputCells::usage = "CountNumberOfLinesOfInputCells[dir] counts the number of lines (with default line length 78) of input cells (at top level) of all notebooks in dir. The operation runs completely in a Mathematica kernel, using only MakeExpression and some fixes thereof (e.g. input cells are splitted (like Ctrl Shift M) programmatically at newlines, since MakeExpression might not work in such a case."; Options[CountNumberOfLinesOfInputCells] = {PageWidth -> 78}; CountNumberOfLinesOfInputCells[dirs__, OptionsPattern[]] := Module[ {files, n}, files = FileNames["*.nb", dirs]; Print["counting lines of ", Length[files], " files in ", Riffle[Flatten[{dirs}], " "]]; (*TODO: figure out why ParallelTable does not seem to work here; maybe Streams do not parallelize well? *) Total[Monitor[Table[SlocSingleNB[files[[n]], OptionValue[PageWidth]], {n, 1, Length[files]}], n]] ]; SlocSingleNB[(f_String)?FileExistsQ, pagewidth_:78] := Quiet[Module[ {inputcells, holdcompletelist, o, tmpfile, le}, inputcells = (Cases[#1, Cell[_BoxData, "Input", ___]] & )[ Flatten[Replace[Get[f], Notebook[z_, ___] :> z] //. Cell[CellGroupData[{c__Cell}, _]] :> c]]; (* Split inputcells at newlines. This is done because MakeExpression[#,StandardForm]& @ BoxData[{ RowBox[{ RowBox[{"SetAttributes", "[", RowBox[{"f", ",", "Listable"}], "]"}], ";"}], "\n", RowBox[{ RowBox[{"f", "[", "x_", "]"}], ":=", RowBox[{"x", "^", "2"}]}]}] does not work. This cell can be found in Defintion.nb *) inputcells = inputcells //. {Cell[BoxData[{r1__, "\n".., r2__}], "Input", ___] :> Sequence[Cell[ BoxData[{r1}] , "Input"], Cell[BoxData[{r2}], "Input"]], Cell[BoxData[{r1__, "\[IndentingNewLine]".., r2__}], "Input"] :> Sequence[Cell[BoxData[{r1}], "Input"], Cell[BoxData[{r2}], "Input"]]}; (* something is weird with Defition: e.g. MakeExpression[ RowBox[{"Definition", "[", "f", "]"}], StandardForm] returns HoldComplete[Null] Substitution Definition by Identity[Definition] seems to work. *) If[ !FreeQ[inputcells, "Definition"], inputcells = inputcells /. "Definition" -> "Identity[Definition]" ]; holdcompletelist = (MakeExpression[First[#1], StandardForm] //. ErrorBox[err_] :> Sequence[] & ) /@ inputcells; o = OpenWrite[]; Do[WriteString[o, (StringJoin[ToString[#1, InputForm, PageWidth -> pagewidth], ";\n"] & )[Unevaluated @@ holdcompletelist[[i]]]], {i, Length[holdcompletelist]}]; tmpfile = Close[o]; le = Length[Import[tmpfile, "Lines"]]; DeleteFile[tmpfile]; le ]]; (* This takes a couple of seconds and should count 1690 lines of code of 227 notebooks: AbsoluteTiming[CountNumberOfLinesOfInputCells[FileNameJoin@{$InstallationDirectory,"AddOns"},Infinity]]

This takes about 10 minutes and counts 13395865 lines of code of 10284 notebooks:
AbsoluteTiming[CountNumberOfLinesOfInputCells[\$InstallationDirectory, Infinity]]
*)
-