2
$\begingroup$

Say I have a function f[x_,...]:=... in a *.wl-Package form that depends on a bunch of Packages (some other *.wl files), and say I want to call this function f several times in parallel. I am currently using a set-up of the following form

LaunchKernels[2];
ParallelNeeds["Privatef`",".../f_file.wl"];

ParallelTable[
output[i]=f[i];
,{i,1,10}];

where i is just supposed to represent a set of parameters. The function f itself is of the form

BeginPackage["Privatef`"]

f

Begin["fenv`"]
Needs["other Packages"]

some code

End[]

EndPackage[]

Now, my problem is that the function f is distributed into the different subkernels I generated, though, the other Packages don't seem to load in each subkernel. I have tried to also use ParallelNeeds on these additional Packages, but without success. So how do I get Mathematica to load each additional Package that f requires in each subkernel that was launched?

$\endgroup$
9
$\begingroup$

Are the packages you need in the $Path?

If not, the following may be the simplest solution:

  • Create a package file (.m or .wl) which simply modifies the $Path.

    BeginPackage["PathLoader`"]
    $Path = Join[$Path, {"/path/one", "/path/two"}];
    EndPackage[]
    

    The only purpose of the BeginPackage/EndPackage here is to ensure that (Parallel)Needs won't complain about "context not created".

  • Put this in a standard location, e.g. $UserBaseDirectory/Applications/PathLoader.m.

  • Start your session with

    Needs["PathLoader`"]
    ParallelNeeds["PathLoader`"]
    

Now $Path will contain all your directories both on the main kernel and on the subkernels.

I use similar (but more complicated) workarounds for Kernel/init.m not being loaded on any subkernels.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.