I have a notebook with about 20 initialization cells. Evaluating all of them takes a few minutes. Mathematica uses only one core for this. In my case, the cells are all independent so it should be easy to parallelize the process. Is there any way to tell Mathematica to use all cores to evaluate whatever initialization cell is next in line?

  • 1
    $\begingroup$ No, there isn't. You have to rewrite the initialization code and parallelize it yourself. You would be using functions like ParallelSubmit. It will be your responsibility to ensure that any definitions (variables, etc.) are present on the main kernel, and not only on a specific subkernel. $\endgroup$ – Szabolcs Dec 12 '16 at 10:21
  • $\begingroup$ Depending on what you are doing you can sometimes pre-run some of the longer initializations and save the output. Then all you need to do is load the result at initialization time. $\endgroup$ – Ramble Dec 12 '16 at 17:36
  • $\begingroup$ @Ramble Would such output be saved in an external file? Could you give me some kind of keyword to search for additional information? $\endgroup$ – Casimir Dec 13 '16 at 13:56
  • 1
    $\begingroup$ You create a new file to run your standard initializations and Export them all. Its a standard Export/Import, you just need to be careful of the format. mx.gz is generally fine though I have used h5 for massive datasets. Most initialization is just loading an existing dataset, preliminary output or 1/0's. $\endgroup$ – Ramble Dec 13 '16 at 15:10
  • 1
    $\begingroup$ Parallel is a really mixed bag because so many standard functions are not optimized. MMA uses math libraries that are sometimes really well optimized for large matrices already (Parallel can kill that efficiency). Sometimes further efficiency is gained by combined several initializations into one larger matrice. You can then "slice" off chunks to use in various parts of your main program. Also look for builtin elements that are designed to create large random datasets in one command. $\endgroup$ – Ramble Dec 13 '16 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.