11
$\begingroup$

How can I automatically count the number of lines of code "Input" existing inside all the notebook files contained in a specified folder?

I know there is the issue of considering or not the automatic line breakings as new lines. But considering the purpose of this exercise (statistically convince that the amount of developed work on Mathematica justifies keeping a license), I'm not searching for exact values.

$\endgroup$
4
  • 3
    $\begingroup$ If this is a metric for the volume of code, perhaps LeafCount is better for Mathematica. What do you think? Mathematica doesn't respect line breaks and tends to put everything on the same line when converting between e.g. InputForm and StandardForm. $\endgroup$
    – Szabolcs
    Feb 13, 2012 at 18:32
  • $\begingroup$ Given the goal,wouldn't a page count do as well? In that case you could use Heike's code here. Or convert the page count to a line count by finding out how many lines fit on a page. $\endgroup$ Feb 13, 2012 at 19:00
  • $\begingroup$ Sorry for taking so long to come back... This is to compare with transforming all code to MATLAB, or a standard .NET code. This means that it has to be apples with apples; and I think the most typical are the kloc (lines of code) $\endgroup$
    – P. Fonseca
    Feb 13, 2012 at 20:07
  • 1
    $\begingroup$ @Szabolcs, related Q: rather than worrying about line breaks, how to count characters in input expressions (including if they contain quotes) and divide by 128 to obtain LOC? I mention possible quotes b/c initially wanted to wrap input in quotes to take StringLength $\endgroup$ Jun 27, 2018 at 22:02

5 Answers 5

8
$\begingroup$

Assuming that the notebook object is contained in the variable nb (for the current notebook do nb = EvaluationNotebook[]), one could use the following:

StringCount[
 First@FrontEndExecute@FrontEnd`ExportPacket[
    Notebook@Cases[First@NotebookGet[nb], Cell[_, "Input"|"Code", ___]],
    "PlainText"],
 "\n"
 ]

Using "InputText" will wrap lines much more frequently and will give a substantially larger count.

This is equivalent to Alt-clicking an input cell to select all input cells, then copying the contents as "plain text" or "input text" and counting the lines.

As you can see, Mathematica does not really respect line breaks. If you are trying to measure the volume of code, I suggest using the LeafCount of expressions instead of counting code lines.

$\endgroup$
5
  • $\begingroup$ @Leonid This is the same as copying the code using the context menu, so it will never contain any \[IndentingNewLine]. Did you find what is causing the errors in those notebooks? $\endgroup$
    – Szabolcs
    Feb 13, 2012 at 19:16
  • $\begingroup$ Ok, that makes sense - probably I was wrong about new lines then. Looks like those errors are caused by some escaped new lines. I am getting errors like this: "String or list of strings expected at position [NoBreak]1[NoBreak] \in [NoBreak]StringCount[First[$Failed],\!(\"\\\\n\")]" $\endgroup$ Feb 13, 2012 at 19:48
  • $\begingroup$ Regarding line breaks: I think I know what you meant, but literally speaking, Mathematica does respect linebreaks. In oher words, at any given time you may check what the box expression is, and every time you see there "\n" or "[IndentingNewLine]", you can be sure there will be a linebreak. Sometimes, Mathematica breaks excessively long lines without inserting linebreaks, just as other editors do - but those should not be considered linebreaks anyway, I think. $\endgroup$ Feb 13, 2012 at 20:02
  • $\begingroup$ @Leonid What I meant was that when you paste code around (paste it to a text editor, then back, or paste from a website to Mathematica), line breaks get shuffled around. Also converting a cell to e.g. StandardForm (ctrl-shift-n) will eat linebreaks. $\endgroup$
    – Szabolcs
    Feb 13, 2012 at 20:10
  • $\begingroup$ Ok, I see - just didn't get what you meant. There seems to be a real impedance mismatch between the FE interface (notebooks) and code editing /coding, as we are used to, in other languages, and this is just one facet of it. I think, often it negatively affects productivity, except in the earliest exploratory stage. I am actually inclined to move to "Program"-style cells also for Mathematica code, probably will do this some time soon. $\endgroup$ Feb 13, 2012 at 20:20
8
$\begingroup$

Here is my version:

ClearAll[nbLines];
 nbLines[nb_NotebookObject] :=
     With[{inputs =
         Cases[NotebookGet[nb], Cell[BoxData[data_], "Input", ___] :> data, Infinity]
       },
      Length[inputs] + 
      Count[Characters@StringJoin@Cases[inputs, _String, Infinity], 
        "\n" | "\[IndentingNewLine]"]
     ]

This may not be precise (because the method of input extraction is not totally robust), but should be good enough for qualitative estimate.

$\endgroup$
0
5
$\begingroup$

Somehow I do like kernel-only solutions: (* see also the github repository here *)

CountNumberOfLinesOfInputCells::usage = "CountNumberOfLinesOfInputCells[dir] counts the number of lines (with default 
line length 78) of input cells (at top level) of all notebooks in dir. The operation runs completely in a Mathematica kernel, using only MakeExpression and 
some fixes thereof (e.g. input cells are splitted (like Ctrl Shift M) programmatically at newlines, since MakeExpression might not work in such a case.";

Options[CountNumberOfLinesOfInputCells] = {PageWidth -> 78}; 
CountNumberOfLinesOfInputCells[dirs__, OptionsPattern[]] :=
    Module[ {files, n},
        files = FileNames["*.nb", dirs];
        Print["counting lines of  ", 
        Length[files], " files in ", Riffle[Flatten[{dirs}], " "]];
        (*TODO: figure out why ParallelTable does not seem to work here; maybe Streams do not parallelize well? *)
        Total[Monitor[Table[SlocSingleNB[files[[n]], OptionValue[PageWidth]], 
           {n, 1, Length[files]}], n]]
    ]; 
SlocSingleNB[(f_String)?FileExistsQ, pagewidth_:78] :=
    Quiet[Module[ {inputcells, holdcompletelist, o, tmpfile, le},
              inputcells = (Cases[#1, Cell[_BoxData, "Input", ___]] & )[
                 Flatten[Replace[Get[f], Notebook[z_, ___] :> z] //. 
                   Cell[CellGroupData[{c__Cell}, _]] :> c]];
              (* Split inputcells at newlines. This is done because 
              MakeExpression[#,StandardForm]& @ BoxData[{ RowBox[{ RowBox[{"SetAttributes", "[", RowBox[{"f", ",", "Listable"}], "]"}], ";"}], "\n", 
                                                          RowBox[{ RowBox[{"f", "[", "x_", "]"}], ":=", RowBox[{"x", "^", "2"}]}]}] 
              does not work. This cell can be found in Defintion.nb *)
              inputcells = 
              inputcells //. {Cell[BoxData[{r1__, "\n".., r2__}], "Input", ___] :> Sequence[Cell[
              BoxData[{r1}]
              , "Input"], Cell[BoxData[{r2}], "Input"]], Cell[BoxData[{r1__, "\[IndentingNewLine]".., r2__}], 
              "Input"] :> Sequence[Cell[BoxData[{r1}], "Input"], Cell[BoxData[{r2}], "Input"]]};
              (* something is weird with Defition: e.g. 
                  MakeExpression[ RowBox[{"Definition", "[", "f", "]"}], StandardForm]
                  returns HoldComplete[Null]
                  Substitution Definition by Identity[Definition] seems to work.
              *)
              If[ !FreeQ[inputcells, "Definition"],
                  inputcells = inputcells /. 
                  "Definition" -> "Identity[Definition]"
              ];
              holdcompletelist = 
              (MakeExpression[First[#1], StandardForm] //. ErrorBox[err_] :> Sequence[] & ) /@ 
              inputcells;
              o = OpenWrite[];
              Do[WriteString[o, (StringJoin[ToString[#1, InputForm, PageWidth -> pagewidth], 
                   ";\n"] & )[Unevaluated @@ holdcompletelist[[i]]]], {i, Length[holdcompletelist]}];
              tmpfile = Close[o];
              le = Length[Import[tmpfile, "Lines"]];
              DeleteFile[tmpfile];
              le
          ]]; 
 (*
 This takes a couple of seconds and should count 1690 lines of code of 227 notebooks:

  AbsoluteTiming[CountNumberOfLinesOfInputCells[FileNameJoin@{$InstallationDirectory,"AddOns"},Infinity]]

This takes about 10 minutes and counts 13395865 lines of code of 10284 notebooks:
 AbsoluteTiming[CountNumberOfLinesOfInputCells[$InstallationDirectory, Infinity]]
*)
$\endgroup$
4
$\begingroup$

If the notebook contains only input the following rough approach using Import will do:

StringCases[Import["ExampleData/document.nb", "Plaintext"] ,    
  Longest[ {"\n" | "\r"} ..]] // Length

(*
==> 19
*)

If there are output cells with lines of text (graphics output was already removed by the Plaintext import), we can remove the output cells first, export to string as NB and re-import as plain text counting lines of input:

  StringCases[
  ImportString[
   ExportString[
    Import["ExampleData/document.nb"] /. 
     Cell[___, "Output", ___] -> Sequence[],
    "NB"
    ],
   "Plaintext"
   ],    Longest[ {"\n" | "\r"} ..]
  ] // Length

(*
==> 15
*)
$\endgroup$
1
$\begingroup$

Using NotebookImport[] (introduced in 2015, version 10.1) simplifies things a lot:

nbinputlinecount = Length@NotebookImport[notebookpath, "Input"]

where notebookpath is the file path to the desired notebook. This only counts the number of lines of input. However, input lines can have multiple lines within them. To tally the total number of lines within all the input cells use the following:

linesperinput[inputline_] := 
   Length@Select[
       ToString@Extract[inputline, #, Hold] & /@ 
          Range[Depth[inputline] - 1], # != "Hold[Null]" &]

anb = NotebookImport[notebookpath, "Input"]
Total[linesperinput[#] & /@ anb]
$\endgroup$
2
  • 1
    $\begingroup$ This just gives the number of "Input" cells I think... $\endgroup$
    – b3m2a1
    Feb 4, 2020 at 23:26
  • $\begingroup$ You are right. The read in from notebookimport would have to be parsed for new lines to determine lines within the input cells. I will update it when I get a chance $\endgroup$
    – boobami
    Feb 6, 2020 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.