Logging intermediate results to notebook

Question

I would like to add the following logging functionality to a package I'm writing:

1. For functions whose (intermediate) results could be important there should be an option to let them write these results to a log file. Here

   1. The log file would be a notebook file. This way I'm sure anyone using mathematica can open it, and one can harness the power of notebooks later on if needed.
   2. The functions would write cells to this file with (a) text containing info and (b) hyperlinks to separate notebooks. This way the log remains readable and one can click on the hyperlinks of choice to open a notebook with the requested results.

2. There should be a PrintLog[ code_, directory_ ] function that can take any code and turns on logging for the functions for which I implemented logging.

An answer to the question Adding logging functionallity in package proposes to insert printLog statements throughout the code, (where printLog is just a symbol with attribute HoldAllComplete) and then running the code inside a Block statement which replaces printLog with a custom function (say myPrint).

For example, if one defines the function MeanPlus, which calls the function NMean as follows:

SetAttributes[ printLog, "HoldAllComplete" ];

Options[NMeanPlus] = { "AddTo" -> 0 };
NMeanPlus[ l_, OptionsPattern[]] :=
    Module[{ ov = OptionValue["AddTo"], procID = ToString @ Unique[], result },
        printLog[ "NMeanPlus:Start", { procID, l, ov } ];
        
        result = NMean[ l ] + OptionValue["AddTo"] ;
        
        printLog[ "NMeanPlus:End", { procID, result } ];
        
        result
    ];

NMean[ l_ ] := 
    Module[ { result, procID },
        printLog[ "NMean:Start", { procID, l } ];
        
        result = Mean @ N @ l;
        
        printLog[ "NMean:End", { procID, result } ];
        
        result
    ];  

Then we can add specific print statements for each phase of the evaluation:

(* Specific print statement to add cells to the notebook with fileName fn *)
myPrint[ dir_, fn_ ][ "NMeanPlus:Start", { procID_, l_, ov_ } ] :=
    (* Code to add cell to notebook *);
myPrint[ dir_, fn_ ][ "NMeanPlus:End", { procID_, result_ } ] :=
    (* Code to add cell to notebook *);
myPrint[ dir_, fn_ ][ "NMean:Start", { procID_, l_ } ] :=
    (* Code to add cell to notebook *);
myPrint[ dir_, fn_ ][ "NMean:End", { procID_, result_ } ] :=
    (* Code to add cell to notebook *);

and define a PrintLog around the following lines:

(* Function that replaces the printLog symbols by our custom myPrint in the code *)
PrintLog[ code_, dir_ ] :=
    With[{ nbFileName = ..., nbStorageDirectory = ... },

        createNoteBook[ nbFileName ];
        createStorageDirectory[ nbStorageDirectory ];

        Block[{ printLog = myPrint[ nbStorageDirectory, nbFileName ] },
            code
        ]
    ];

My specific problem is now: how do I implement the myPrint function such that it adds cells to an existing notebook. This notebook would be created by the PrintLog function on the hard drive and incrementally updated in case the evaluation gets interrupted. The cells should contain hyperlinks to data files stored in a separate directory which is also created by the PrintLog function.

An example of how such a notebook could look like would be:

At the moment my idea was to create a text file and add the cells as strings, but the code is super ugly and hard to read/debug/expand. I have looked at the help files for notebook manipulation, but I am swamped by the number of different approaches to manipulating notebooks. Most functions also seem to revolve around front-end notebook objects rather than notebook files.

Answer 1

The following code will give you a starting point, it should print the input and output into any given notebook with some time and date information:

ClearAll[myPrint];

SetAttributes[myPrint, HoldFirst];
myPrint[code_, directory_?(FileExistsQ), opt : OptionsPattern[]] :=
 Module[
  {logNB, outputCode = ToBoxes[code]},
  logNB = NotebookOpen[directory, Visible -> True];
  NotebookWrite[logNB, 
   Cell["Log Item - " <> DateString[] <> " :", "Item"]];
  NotebookWrite[logNB, Cell[BoxData[MakeBoxes[code]], "Input"]];
  NotebookWrite[logNB, Cell[BoxData@outputCode, "Output"]];
  MakeExpression@outputCode /. HoldComplete[expr_] -> expr
  ]

(* Error and messages *)
myPrint::"MissingLogFile" = "Failed to locate Log notebook in `1`";
myPrint[_, directory_] := (
  Message[myPrint::"MissingLogFile", directory];
  Abort[]
  )

Example use case and output:

Ideally, the function should also return the output expression. So for example the following expression should be valid and will return 102 logging 100 into the log notebook:

myPrint[100, FileNameJoin@{NotebookDirectory[], "logNB.nb"}] + 2