I am trying to do automated testing of my MMA package by launching test cases that are generated on-the-fly with GenerateDocument and a template document, and then evaluated with NotebookEvaluate. I am trying to do this with LinkLaunch/LinkRead/LinkWrite and have followed e.g. Restarting Mathematica automatically.

The idea is to have a single MMA notebook that can launch multiple tests of a package (varying the various options, for example) over multiple kernels in parallel, and collect the results when the runs are done. GenerateDocument is very nice because it can generate a new notebook from a template notebook in which pre-defined fields have been filled from an Association via GenerateDocument's second argument. My plan was to then submit this generated document to a LinkLaunch-ed kernel for evaluation

However I can almost never get a proper evaluation of these calls.

Is this a bug or am I just doing something wrong with the various Link-commands?

I suspect that it is a file-permissions issue - that GenerateDocument locks some common system file (like perhaps a stylesheet) such that subsequent instances are blocked. The errors are varied and random, which suggests to me a race-condition on some files, but then who knows. Certainly the most common errors are all file related (DeleteFile, FileFormatQ).

The error messages/responses are all seemingly from steps A4 and B4 and are, inter alia:

  • MessagePacket[NotebookEvaluate,nbnsvd].

  • MessagePacket[DeleteFile,privv].

  • MessagePacket[ImportExport`FileFormatQ,nffil].

  • MessagePacket[Import,nffil].

  • No response - eternal hang/block.

  • Popups of multiple windows.

  • Popup of a DOS command window.

  • Creation of multiple mathematica.exe kernels after killing one or more kernels with Taskmanager.

  • 100% CPU usage for long durations.

Here is the code:


$Version<>": Launching 2 links at "<>DateString["Time"]<>"..."

"A1: LinkLaunch and reading of initial prompt:"
link[1] = LinkLaunch["\"" <> First[$CommandLine] <> "\" -mathlink"]
(*read/drain initial prompt:*)

(*Do it again:*)
"B1: LinkLaunch and reading of initial prompt:"
link[2] = LinkLaunch["\"" <> First[$CommandLine] <> "\" -mathlink"]
(*read/drain initial prompt:*)

"A2: Request and Read $ProcessID:"
LinkWrite[link[1], Unevaluated@$ProcessID]

"B2: Request and Read $ProcessID:"
LinkWrite[link[2], Unevaluated@$ProcessID]

"A3: Evaluate external notebook from template:"
LinkWrite[link[1], Unevaluated@EvaluatePacket[ UsingFrontEnd[ nb = GenerateDocument[ "C:\\TEMP\\SimpleTestTemplate.A.nb", <|"DataTicks" -> True|>]; NotebookEvaluate[n b, InsertResults -> True]]]]

"B3: Evaluate external notebook from template:"
LinkWrite[link[2], Unevaluated@EvaluatePacket[ UsingFrontEnd[ nb = GenerateDocument[ "C:\\TEMP\\SimpleTestTemplate.B.nb",<|"DataTicks" -> False|>]; NotebookEvaluate[nb, InsertResults -> True]]]]

"A4: LinkRead Awaiting results on link[1] at "<>DateString["Time"]<>"..."

"A5: Quitting and Closing link[1] at "<>DateString["Time"]<>"..."
LinkWrite[link[1], Unevaluated@EvaluatePacket[Quit[]]]

"B4: LinkRead Awaiting results on link[2] at "<>DateString["Time"]<>"..."

"B5: Quitting and Closing link[2] at "<>DateString["Time"]<>"..."
LinkWrite[link[2], Unevaluated@EvaluatePacket[Quit[]]]

"FIN: all closed at "<>DateString["Time"]<>"."

The usual (bad) behavior is to get one or even both of the correct outputs on the first evaluation of the above code, but for all subsequent evaluations to have one or both link-evaluations fail at the A4 or B4 steps. The desired and correct behavior is to return two slightly different PairwiseScatterPlots. I use PairwiseScatterPlots because my particular use-case also involves loading a package.

If I just replace the GenerateDocument in steps A3 and B3 with something non-file-based like


instead of the

nb = GenerateDocument[
  <|"DataTicks" -> True|>];
NotebookEvaluate[nb, InsertResults -> True]

then everything goes through fine, even under repeated and frequent testing.

The SimpleTestTemplate.nb files are nothing more than two lines in a 'Template Notebook':


PairwiseScatterPlot[Table[{x, Sin[x], Cos[x]}, {x, 0, 2 Pi, 0.1}], DataTicks -> "DataTicks"]

Which can be created by going to File > New > Template Notebook. The "DataTicks" field has to be selected/highlighted and set as a 'Named' field by clicking the 'slot' button (the first drop-down button on the left, which has 'Named' and 'Positional' as the two options) in the Template Notebook. I created two copies of this file ('A' and 'B') thinking this might be the file with the permissions conflict, but it wasn't. I then placed these files in the "C:\TEMP\" directory.

I had considered using e.g. ParallelEvaluate for this, but I expect that in fact the problem with GenerateDocument will be the same, so am going ahead and asking this question as it stands, with LinkRead et al.

I am doing all of this on Windows 8.1 with MMA 11.3.

Thanks for any help! I am also forwarding this to Wolfram as a bug report.


The workaround, I realized after posting, is to simply put a Pause[3] between steps A3 and B3 which presumably gives the file-system enough time to release whatever locks the first call has placed on the unknown file.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.