9
$\begingroup$

I have ~150 student-submitted Mathematica notebooks for an assessed assignment. While I've been marking them, I'm suspecting there is a reasonable amount of plagiarism going on, when multiple students make the same odd errors throughout.

When I find something odd, I can use Notebook++ to search through all the notebooks for where it crops up elsewhere, but the .nb files have a lot of extra RowBox and BoxData etc.

I'm looking to convert just the input cells from each notebook into a text file, is there a way to do that programmatically?

The answer at: Converting a notebook to plain text programmatically shows how to automate the process of saving a single notebook as a text file, but I want just the "Input" Cells and can't figure out how to do that.

Edit:

The answer from Alexey works generally, except when an integral has been input using the Integral sign on the palette. Running the code:

name = "test2.nb"
Export["test.txt", StringJoin[
  Riffle[NotebookImport[name, "Input" -> "Text"], 
   "\n=================\n"]], "Text",  CharacterEncoding -> "PrintableASCII"]

then makes something strange happen involving a SubsuperscriptBox that I can't fix, which makes the output go over two columns, but within the text file:

Text file from the test notebook

$\endgroup$
3
  • 3
    $\begingroup$ NotebookImport does what you need. This question is quite close to what you want at the end: Export of all notebook files as text files $\endgroup$
    – Kuba
    Commented May 16, 2017 at 16:20
  • $\begingroup$ KraZug, let me know if there is anything that the linked answer didn't cover. $\endgroup$
    – Kuba
    Commented May 17, 2017 at 6:05
  • $\begingroup$ Thanks @Kuba, that is close. Any idea on the strange integral sign behaviour I've just added? $\endgroup$
    – SPPearce
    Commented May 17, 2017 at 11:13

3 Answers 3

6
$\begingroup$

Updated to support multiline input cells and syntax errors

I would avoid linear syntax. Why not use something like the following:

inputCellsToText[nb_, textFile_] := Internal`InheritedBlock[
    {SequenceForm},
    SetAttributes[SequenceForm, HoldAll];
    
    Export[
        textFile,
        Flatten[extractInput /@ NotebookImport[nb, "Input"->"HeldInterpretedCell"]],
        "Text"
    ]
]

extractInput[HoldComplete[ExpressionCell[a_List, __]]] := DeleteCases[
    SequenceForm /@ Unevaluated[a],
    SequenceForm[Null]
]
extractInput[HoldComplete[ExpressionCell[a_, __]]] := SequenceForm[a]

For example, the following notebook:

nb = Uncompress @"1:eJzVVc1rE0EUj7WxftBaD4onWQ9KQmrNbkIa24gkaSJVYyRbVMTLJp1tR9OZMDtrGw/iWfQqFRT8I6SXinryIChexVJQ60F6Kd48CM6bbD42TfMh9eDlzb437/P33rw9nqc50+vxeKy9glymHOUpvW3uBkm/IJewxc1dVS6JikWzD7hBhzvPqF2aNLjRZNLvMpEOBgRJ0EWp7N3qf48gObogNCoRarfS8SFBdDtv2SXErALDJS4UdbC5Oa5pWkIHH8GaQE11cCq5A4KkmVHgmBJwBybqFkNXYVLH14WOKmmgEmhIZu9KHW4rVJPU3z5hp7L0WDjpmDqCUDzURTaVGAHnu42pG6XOjsOSKg3fAR2aO0VKNq97y9lFZB10xiE5Z5BZNI3nkVWfE/CIwB5s7iRuodWljfijh5/n4KxdrCwPvVoXgsfvIq/hZM+9A2/XGzU+Dg++B8Fmjn6AsymHw+IjNV+aMyx8F+llwo3FFGOUWVJlmtmoXmTr0W05ORVEg6FksAHirM27AkHmDgrjR3/Lct/E78vqTi+dkLVMfJk4+x1qaZ+a04QdDR0bj8nQ0ku2hEh3y6DFC94e0TYPvPmVhHoZLgQa1bn4ufltBQp80Bd+udoZy96S6gXyOoCADVuO3ZD5/EuA2z7jbTdUdaYTauMK2aH1mK+t6mBUCgqVLfjX/c18+vFkTeC4ce/rs7Uu+vs/YdLTi24E5djJF08BjCNXfklQ6kPm9rNffFzDZIYu6GItuodrChKNBqPyjETPNJkO1kwzBpvFxGr1o61yOCYys/YJLm5zOm9wXHAruO+wJrgWJacZJTxFZq6K1ogFrJ8TMlUdVRWTMiVjFJSsrlxXFqMRxRfSTuUxH1EiYTiVi4gRVPQrvgsGsQ1WVsIjihZUx/xNUYah9bxcRJPIxATDmrd0QEnwhl3koyT/ByPIIdA=";

enter image description here

produces the following text file:

Integrate[1/(1 + x^2), {x, 0, Infinity}]

2 + 2

ErrorBox[ErrorBox[RowBox[{RowBox[{"4", " ", "4"}], "+"}]]]

Subscript[x, 2]^3

α^2 + (Element[b, c])

$\endgroup$
6
  • $\begingroup$ This works really nicely, except for if there are multiple lines in a single Input Cell they are concatenated together with Null. Adding //. HoldComplete[a___, Null, b___] -> Sequence[HoldComplete[a], HoldComplete[b]] before swapping HoldComplete with SequenceForm solves that. One more wrinkle, if the input line gives a syntax error then you get a Failure[] message. $\endgroup$
    – SPPearce
    Commented May 17, 2017 at 18:43
  • 2
    $\begingroup$ @KraZug I updated my answer to use "HeldInterpretedCell" instead so that you can still get syntactically correct lines of input, and I removed the Null objects. $\endgroup$
    – Carl Woll
    Commented May 17, 2017 at 20:07
  • $\begingroup$ This is far quicker to run than Alexey's current version, but is giving me ErrorBox's even sometimes when I don't see why. I'll try and get an example sorted later. $\endgroup$
    – SPPearce
    Commented May 18, 2017 at 10:39
  • $\begingroup$ I've just asked a question as to why NotebookImport sometimes fails to import when the code is spread over multiple lines. mathematica.stackexchange.com/questions/146415/… $\endgroup$
    – SPPearce
    Commented May 19, 2017 at 12:41
  • $\begingroup$ Is it possible to keep comments that are in the input cells as well? Right now, all comments are stripped out. For example, if I have a notebook with this one input cell (*this is a comment*) x=1; and run your code on the notebook, the output .txt file will only have x=1; in it. It will be very useful to keep comments if possible. Thanks $\endgroup$
    – Nasser
    Commented Feb 24, 2018 at 18:49
7
$\begingroup$

As Kuba say in the comments, NotebookImport does what you need. Specifically, NotebookImport[nb, "Input" -> "Text"] preserves the original formatting and syntax of input cells what should fit your purposes exactly. You can Riffle the obtained list of strings with some delimiter in order to distinguish different cells in the exported file:

StringJoin[Riffle[NotebookImport[nb, _ -> "Text"], "\n=================\n"]]

It is easy to automatize the conversion:

nbFileNames = FileNames["*.nb"];
Do[
 Export[name <> ".txt",
  StringJoin[Riffle[NotebookImport[name, _ -> "Text"], "\n=================\n"]]
  , "Text"],
 {name, nbFileNames}]

UPDATE

Here is an improved solution which correctly handles special and Unicode characters as well as 2D typesetting, and encodes everything using the linear syntax (tutorial/StringRepresentationOfBoxes):

nbFileNames = FileNames["*.nb"];

Do[Export[name <> ".txt",
  StringJoin[Riffle[
    ToString[#, InputForm, CharacterEncoding -> "PrintableASCII"] & /@
     NotebookImport[name, "Input" -> "Text"],
    "\n=================\n"]],
  "Text"],
 {name, nbFileNames}]

The linear syntax has an advantage of being universal, exact and platform-independent representation of a string (which is allowed to contain rich formatting and 2D typesetting) at the expense of substantially lesser readability.

$\endgroup$
8
  • $\begingroup$ Thank you. With the specification of "Input" in this code, this generally works. It does need CharacterEncoding -> "PrintableASCII" on the Export to deal with special characters, but I'm having some issues with formatting when there is an Integral sign in the code. See my edit to the question. $\endgroup$
    – SPPearce
    Commented May 17, 2017 at 11:13
  • $\begingroup$ @KraZug I see, would that be acceptable? Export[..., ToString[..., InputForm, CharacterEncoding -> "PrintableASCII"], "Text" ]? $\endgroup$
    – Kuba
    Commented May 17, 2017 at 11:30
  • $\begingroup$ The answer (with the modifications mentioned above, I wasn't sure whether I should just edit them into in) works nicely in general, but fails for the Integral symbol for some reason. I can't get ToString[..., InputForm] to work and give the same kind of result. $\endgroup$
    – SPPearce
    Commented May 17, 2017 at 11:41
  • $\begingroup$ @KraZug please elaborate, what do you expect from the 2D representation of integral in a text file? $\endgroup$
    – Kuba
    Commented May 17, 2017 at 12:08
  • $\begingroup$ I would be happy for whatever is appropriate, provided it doesn't screw up the rest of the file. It makes the lines after the integral go into two columns in the .txt file $\endgroup$
    – SPPearce
    Commented May 17, 2017 at 12:42
3
$\begingroup$

Here is another approach which offers better readability while maintaining robustness of the method shown in the "UPDATE" section of previous answer (with only exception to compatibility between different platforms: carriage returns aren't escaped here).

In order to avoid non-ASCII characters in the output file I use here improved fromCode function from this answer (which converts character codes into Mathematica's ASCII representation of the corresponding characters):

fromCode[c_Integer] /; c <= 127 := FromCharacterCode[c];
fromCode[c_Integer] := 
  StringTake[ToString[FromCharacterCode[c], InputForm, 
    CharacterEncoding -> "PrintableASCII"], {2, -2}];

The main function:

convertNBtoTXT[fileName_String] :=
  Module[{stream, lines, delim = "\n===============\n"},
   stream = OpenWrite[fileName <> ".txt", BinaryFormat -> True];
   lines = NotebookImport[fileName, "Input" -> "Text"];
   WriteString[stream, Sequence @@ fromCode /@ ToCharacterCode[#], delim] & /@ lines;
   Close[stream];
   ];

Note the Sequence @@ trick: writing character-by-character allows to avoid conversion of typesetting into OutputForm, and as the result the linear syntax is exported exactly as it is present in the string.

Usage:

nbFileNames = FileNames["*.nb"];
convertNBtoTXT /@ nbFileNames;

In my tests it correctly handles Unicode and special characters as well as 2D typesetting.

The original input cells can be recovered from exported ASCII file using FrontEnd`UndocumentedTestFEParserPacket as follows:

fileName = "document.nb.txt";
delim = "\n===============\n";

NotebookPut[
 Notebook[
  Cell[First@
      MathLink`CallFrontEnd[FrontEnd`UndocumentedTestFEParserPacket[#, True]],
     "Input"] & /@
   ReadList[fileName, Record, RecordSeparators -> delim]]]

UPDATE

Using the ExportAsASCII function from this answer we can achieve the same in a more efficient way:

nbFileNames = FileNames["*.nb"];

Do[
 ExportAsASCII[name <> ".txt", 
  StringJoin[Riffle[NotebookImport[name, "Input" -> "Text"], "\n=================\n"]]],
 {name, nbFileNames}]
$\endgroup$
2
  • $\begingroup$ Both this method and Carl's method have pros and cons, but I think Carl's is marginally better for what I want, as it converts the special characters as well, which means I can search more naturally on expressions that include symbols like -> or ==. Provided Wolfram fix the bug with splitting expressions across lines! $\endgroup$
    – SPPearce
    Commented May 22, 2017 at 9:31
  • $\begingroup$ Carl's method is still much faster, 14s instead of 329s to do 150 notebooks. $\endgroup$
    – SPPearce
    Commented May 22, 2017 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.