I have a Mathematica notebook that I have been running on my desktop and am working on moving it over to a high-performance cluster.
One part of the code is a random walk algorithm that uses a matrix "landscape" to determine probabilities at each step. The random walk algorithm will be run millions of times.
I'd like to take full advantage of parallelization by running the random walk separately from the part of the notebook that creates the matrix. The random walks all need to be using the same landscape matrix.
My question is, is it possible to put the matrix in an environment variable that is then shared by the algorithm instances that I launch?
The supercluster is using slurm to manage jobs, so the idea would be to run the script that makes the matrix, then run the random walk algorithm over and over. This would allow me to make full use of the cluster since each instance could access its own cores and ram.