6
$\begingroup$

I have ~150 student-submitted Mathematica notebooks for an assessed assignment. While I've been marking them, I'm suspecting there is a reasonable amount of plagiarism going on, when multiple students make the same odd errors throughout.

When I find something odd, I can use Notebook++ to search through all the notebooks for where it crops up elsewhere, but the .nb files have a lot of extra RowBox and BoxData etc.

I'm looking to convert just the input cells from each notebook into a text file, is there a way to do that programmatically?

The answer at: Converting a notebook to plain text programmatically shows how to automate the process of saving a single notebook as a text file, but I want just the "Input" Cells and can't figure out how to do that.

Edit:

The answer from Alexey works generally, except when an integral has been input using the Integral sign on the palette. Running the code:

name = "test2.nb"
Export["test.txt", StringJoin[
  Riffle[NotebookImport[name, "Input" -> "Text"], 
   "\n=================\n"]], "Text",  CharacterEncoding -> "PrintableASCII"]

then makes something strange happen involving a SubsuperscriptBox that I can't fix, which makes the output go over two columns, but within the text file:

Text file from the test notebook

$\endgroup$
  • 3
    $\begingroup$ NotebookImport does what you need. This question is quite close to what you want at the end: Export of all notebook files as text files $\endgroup$ – Kuba May 16 '17 at 16:20
  • $\begingroup$ KraZug, let me know if there is anything that the linked answer didn't cover. $\endgroup$ – Kuba May 17 '17 at 6:05
  • $\begingroup$ Thanks @Kuba, that is close. Any idea on the strange integral sign behaviour I've just added? $\endgroup$ – KraZug May 17 '17 at 11:13
4
$\begingroup$

Updated to support multiline input cells and syntax errors

I would avoid linear syntax. Why not use something like the following:

inputCellsToText[nb_, textFile_] := Internal`InheritedBlock[
    {SequenceForm},
    SetAttributes[SequenceForm, HoldAll];

    Export[
        textFile,
        Flatten[extractInput /@ NotebookImport[nb, "Input"->"HeldInterpretedCell"]],
        "Text"
    ]
]

extractInput[HoldComplete[ExpressionCell[a_List, __]]] := DeleteCases[
    SequenceForm /@ Unevaluated[a],
    SequenceForm[Null]
]
extractInput[HoldComplete[ExpressionCell[a_, __]]] := SequenceForm[a]

For example, the following notebook:

nb = Uncompress @"1:eJzVVc1rE0EUj7WxftBaD4onWQ9KQmrNbkIa24gkaSJVYyRbVMTLJp1tR9OZMDtrGw/iWfQqFRT8I6SXinryIChexVJQ60F6Kd48CM6bbD42TfMh9eDlzb437/P33rw9nqc50+vxeKy9glymHOUpvW3uBkm/IJewxc1dVS6JikWzD7hBhzvPqF2aNLjRZNLvMpEOBgRJ0EWp7N3qf48gObogNCoRarfS8SFBdDtv2SXErALDJS4UdbC5Oa5pWkIHH8GaQE11cCq5A4KkmVHgmBJwBybqFkNXYVLH14WOKmmgEmhIZu9KHW4rVJPU3z5hp7L0WDjpmDqCUDzURTaVGAHnu42pG6XOjsOSKg3fAR2aO0VKNq97y9lFZB10xiE5Z5BZNI3nkVWfE/CIwB5s7iRuodWljfijh5/n4KxdrCwPvVoXgsfvIq/hZM+9A2/XGzU+Dg++B8Fmjn6AsymHw+IjNV+aMyx8F+llwo3FFGOUWVJlmtmoXmTr0W05ORVEg6FksAHirM27AkHmDgrjR3/Lct/E78vqTi+dkLVMfJk4+x1qaZ+a04QdDR0bj8nQ0ku2hEh3y6DFC94e0TYPvPmVhHoZLgQa1bn4ufltBQp80Bd+udoZy96S6gXyOoCADVuO3ZD5/EuA2z7jbTdUdaYTauMK2aH1mK+t6mBUCgqVLfjX/c18+vFkTeC4ce/rs7Uu+vs/YdLTi24E5djJF08BjCNXfklQ6kPm9rNffFzDZIYu6GItuodrChKNBqPyjETPNJkO1kwzBpvFxGr1o61yOCYys/YJLm5zOm9wXHAruO+wJrgWJacZJTxFZq6K1ogFrJ8TMlUdVRWTMiVjFJSsrlxXFqMRxRfSTuUxH1EiYTiVi4gRVPQrvgsGsQ1WVsIjihZUx/xNUYah9bxcRJPIxATDmrd0QEnwhl3koyT/ByPIIdA=";

enter image description here

produces the following text file:

Integrate[1/(1 + x^2), {x, 0, Infinity}]

2 + 2

ErrorBox[ErrorBox[RowBox[{RowBox[{"4", " ", "4"}], "+"}]]]

Subscript[x, 2]^3

α^2 + (Element[b, c])

$\endgroup$
  • $\begingroup$ This works really nicely, except for if there are multiple lines in a single Input Cell they are concatenated together with Null. Adding //. HoldComplete[a___, Null, b___] -> Sequence[HoldComplete[a], HoldComplete[b]] before swapping HoldComplete with SequenceForm solves that. One more wrinkle, if the input line gives a syntax error then you get a Failure[] message. $\endgroup$ – KraZug May 17 '17 at 18:43
  • 2
    $\begingroup$ @KraZug I updated my answer to use "HeldInterpretedCell" instead so that you can still get syntactically correct lines of input, and I removed the Null objects. $\endgroup$ – Carl Woll May 17 '17 at 20:07
  • $\begingroup$ This is far quicker to run than Alexey's current version, but is giving me ErrorBox's even sometimes when I don't see why. I'll try and get an example sorted later. $\endgroup$ – KraZug May 18 '17 at 10:39
  • $\begingroup$ I've just asked a question as to why NotebookImport sometimes fails to import when the code is spread over multiple lines. mathematica.stackexchange.com/questions/146415/… $\endgroup$ – KraZug May 19 '17 at 12:41
  • $\begingroup$ Is it possible to keep comments that are in the input cells as well? Right now, all comments are stripped out. For example, if I have a notebook with this one input cell (*this is a comment*) x=1; and run your code on the notebook, the output .txt file will only have x=1; in it. It will be very useful to keep comments if possible. Thanks $\endgroup$ – Nasser Feb 24 '18 at 18:49
6
$\begingroup$

As Kuba say in the comments, NotebookImport does what you need. Specifically, NotebookImport[nb, "Input" -> "Text"] preserves the original formatting and syntax of input cells what should fit your purposes exactly. You can Riffle the obtained list of strings with some delimiter in order to distinguish different cells in the exported file:

StringJoin[Riffle[NotebookImport[nb, _ -> "Text"], "\n=================\n"]]

It is easy to automatize the conversion:

nbFileNames = FileNames["*.nb"];
Do[
 Export[name <> ".txt",
  StringJoin[Riffle[NotebookImport[name, _ -> "Text"], "\n=================\n"]]
  , "Text"],
 {name, nbFileNames}]

UPDATE

Here is an improved solution which correctly handles special and Unicode characters as well as 2D typesetting, and encodes everything using the linear syntax (tutorial/StringRepresentationOfBoxes):

nbFileNames = FileNames["*.nb"];

Do[Export[name <> ".txt",
  StringJoin[Riffle[
    ToString[#, InputForm, CharacterEncoding -> "PrintableASCII"] & /@
     NotebookImport[name, "Input" -> "Text"],
    "\n=================\n"]],
  "Text"],
 {name, nbFileNames}]

The linear syntax has an advantage of being universal, exact and platform-independent representation of a string (which is allowed to contain rich formatting and 2D typesetting) at the expense of substantially lesser readability.

$\endgroup$
  • $\begingroup$ Thank you. With the specification of "Input" in this code, this generally works. It does need CharacterEncoding -> "PrintableASCII" on the Export to deal with special characters, but I'm having some issues with formatting when there is an Integral sign in the code. See my edit to the question. $\endgroup$ – KraZug May 17 '17 at 11:13
  • $\begingroup$ @KraZug I see, would that be acceptable? Export[..., ToString[..., InputForm, CharacterEncoding -> "PrintableASCII"], "Text" ]? $\endgroup$ – Kuba May 17 '17 at 11:30
  • $\begingroup$ The answer (with the modifications mentioned above, I wasn't sure whether I should just edit them into in) works nicely in general, but fails for the Integral symbol for some reason. I can't get ToString[..., InputForm] to work and give the same kind of result. $\endgroup$ – KraZug May 17 '17 at 11:41
  • $\begingroup$ @KraZug please elaborate, what do you expect from the 2D representation of integral in a text file? $\endgroup$ – Kuba May 17 '17 at 12:08
  • $\begingroup$ I would be happy for whatever is appropriate, provided it doesn't screw up the rest of the file. It makes the lines after the integral go into two columns in the .txt file $\endgroup$ – KraZug May 17 '17 at 12:42
2
$\begingroup$

Here is another approach which offers better readability while maintaining robustness of the method shown in the "UPDATE" section of previous answer (with only exception to compatibility between different platforms: carriage returns aren't escaped here).

In order to avoid non-ASCII characters in the output file I use here improved fromCode function from this answer (which converts character codes into Mathematica's ASCII representation of the corresponding characters):

fromCode[c_Integer] /; c <= 127 := FromCharacterCode[c];
fromCode[c_Integer] := 
  StringTake[ToString[FromCharacterCode[c], InputForm, 
    CharacterEncoding -> "PrintableASCII"], {2, -2}];

The main function:

convertNBtoTXT[fileName_String] :=
  Module[{stream, lines, delim = "\n===============\n"},
   stream = OpenWrite[fileName <> ".txt", BinaryFormat -> True];
   lines = NotebookImport[fileName, "Input" -> "Text"];
   WriteString[stream, Sequence @@ fromCode /@ ToCharacterCode[#], delim] & /@ lines;
   Close[stream];
   ];

Note the Sequence @@ trick: writing character-by-character allows to avoid conversion of typesetting into OutputForm, and as the result the linear syntax is exported exactly as it is present in the string.

Usage:

nbFileNames = FileNames["*.nb"];
convertNBtoTXT /@ nbFileNames;

In my tests it correctly handles Unicode and special characters as well as 2D typesetting.

The original input cells can be recovered from exported ASCII file using FrontEnd`UndocumentedTestFEParserPacket as follows:

fileName = "document.nb.txt";
delim = "\n===============\n";

NotebookPut[
 Notebook[
  Cell[First@
      MathLink`CallFrontEnd[FrontEnd`UndocumentedTestFEParserPacket[#, True]],
     "Input"] & /@
   ReadList[fileName, Record, RecordSeparators -> delim]]]

UPDATE

Using the ExportAsASCII function from this answer we can achieve the same in a more efficient way:

nbFileNames = FileNames["*.nb"];

Do[
 ExportAsASCII[name <> ".txt", 
  StringJoin[Riffle[NotebookImport[name, "Input" -> "Text"], "\n=================\n"]]],
 {name, nbFileNames}]
$\endgroup$
  • $\begingroup$ Both this method and Carl's method have pros and cons, but I think Carl's is marginally better for what I want, as it converts the special characters as well, which means I can search more naturally on expressions that include symbols like -> or ==. Provided Wolfram fix the bug with splitting expressions across lines! $\endgroup$ – KraZug May 22 '17 at 9:31
  • $\begingroup$ Carl's method is still much faster, 14s instead of 329s to do 150 notebooks. $\endgroup$ – KraZug May 22 '17 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.