How to run a whole mathematica notebook within a for loop?

Question

I have some data sets labelled data1.dat, data2.dat, and so on, and I have written a notebook which will import a single .dat file, generate a plot, and export that into a .pdf.

For example, to generate the plot graph5.pdf from data5.dat, my code is like this.

index = ToString[5];
importfilename = StringJoin["/path/to/data/data", index, ".dat"];
exportfilename = StringJoin["/path/to/graphs/graph", index, ".pdf"];
(** Lines to generate the plot from importfilename **)
Export[exportfilename, %]

Is there any way to write index = ToString[k] and perform a for loop over the variable k so that the whole Notebook will be evaluated for every k?

I know that something similar is possible in MATLAB/Octave, where I can write my script to a file, and then call that script file inside a for loop for different values of k.

Answer 1

Yep.

You can do this no problem. I suggest looking at the documentation on the workflow in NotebookEvaluate[]. For example, create another notebook with the following contents.

Table[ NotebookEvaluate["/path/to/previous/notebook.nb"], {k, k_min, k_max}]

I'd suggest that maybe a simpler way would be to confine operations to a single note book with the following structure

• Initialise
• Loop: read, then process, the plot each data file
• Wrap up

Easier debugging for one, but your kilometerage may vary.

Answer 2

While it is possible to use NotebookEvaluate for such purposes, I think it will be worth to learn the more recommended ways to automate your tasks:

1. you can write scripts in text-only files which can end in either .m or .wl and run these either from the command line or by simply doing Get["/path/to/script"].

2. Even better is to learn how to write functions, then call the functions in a loop. This would look something like:

   exportPlot[n_]:=Module[{importfilename,exportfilename},
      index = ToString[n];       
      importfilename = StringJoin["/path/to/data/data", index, ".dat"];
      exportfilename = StringJoin["/path/to/graphs/graph", index, ".pdf"];
      (** Lines to generate the plot from importfilename **)
      Export[exportfilename, %]
   ];
   
   Do[exportPlot[k],{k,1,10}]
   

Another note: I would strongly suggest to avoid to use % within such code/scripts. It doesn't cost you anything to assign any result to a variable and use that variable in the following code. It will make your code much easier to read and a lot less error prone.

Answer 3

I see an already accepted answer, which I agree with. I am writing this answer as really an extended comment, which I think too long for the usual comment.

I frequently use NotebookEvaluate[] for the purpose described. To place this in context, I offer an example of a data analysis routine. The routine is written as a notebook. It loads several images which are captured as the output of an optical fiber. The analysis involves image processing, conversion to numerical data, and several curve fittings as well as an iterative successive approximation of numerical values. The notebook is many pages in length.

There are many image sets to be evaluated. It would be possible to take the working notebook, delete all output, and merge the cells so as to produce a function which accepted the image file names and produced the result. The function could then me mapped onto a list of image sets to produce a list of results. This what I often do for less extensive computations.

But this involves a lot of work and, what's worse, produces a system that is difficult to maintain. It does happen that a data set breaks the algorithm and fails to evaluate in a useful way. This happens especially in the early part of developing the algorithm. Then its back to the original notebook because the functional form is too difficult to debug.

The alternative is a controlling notebook and a sub-notebook. The original notebook is made into a sub-notebook merely by commenting out the part of the code that defines the filenames of the images to be evaluated. The controlling notebook is simple, often only a few lines. It opens the sub-notebook and then in a loop defines the image file names, one set at a time, to be evaluated. In each iteration of the loop, it sets the filename variables of the sub-notebook to a new set (reading sets from a list) and calls NotebookEvaluate. The sub-notebook runs to completion and, in doing so, sets the output variables to the results of its evaluation. After NotebookEvaluate returns, the controlling notebook stores the results in a list and goes to the next iteration. When all sets have been evaluated, it then performs any desired evaluation of the data as a whole.

This works wonderfully. And the real benefit is that if a data set breaks the algorithm, requiring modification, you can uncomment the code in the sub-notebook that defines the image file names and run the sub-notebook alone on that data set. This means you can evaluate the algorithm one cell at a time, observing the output for each. So you have the advantage of the interactive notebook format.