1
$\begingroup$

I'm looking to use WSTP to make functions defined in a C MPI application available to a remote Mathematica kernel. I've got this working for a non-MPI version of the application, but now wish to have the WSTP backend run on e.g. 8 remote machines, which are parallelising the backend calls. I'm unsure how this could work in principle. I imagine it's not possible (or at least, sensible) to simply launch the WSTP link on each node (which can MP to eachother) and have the MMA kernel link to each, and ignore the returns of all but one node. Is there a better way to achieve this, e.g. having WSTP run on some intermediate machine which can talk to the backend machines?

Additionally, after compiling the WSTP-wrapped code with MPI compilers, I encounter the error during linking on Linux

WSTPlibs/libWSTPi4.36.a: file not recognized: File format not recognized

invoked from

mpicc -O2 -std=c99 -mavx -Wall -Isomefolder -o wstplink filename.o libWSTPi4.36.a -lm

I suppose this is because the libWSTPi4.36.a archive is for MacOS, taken from /Applications/Mathematica.app/Contents/SystemFiles/Links/WSTP. Where can an archive for Linux be downloaded?

Improve this question
$\endgroup$
5
  • $\begingroup$ I don't know exactly how this will work but it definitely is a cool idea. One question: what do you imagine for the "intermediate machine" and how is it different from where you're launching the main process? $\endgroup$
    – b3m2a1
    Jan 16, 2019 at 19:33
  • $\begingroup$ @b3m2a1 Thanks very much! I'm really not sure, since I don't know much about how ObjectLink works (or how the kernel talks to the remote WSTP through TCP). Perhaps the intermediate node would be running an MMA kernel and TCP-talk to all of the 'work' nodes on different ports (or pairs of ports), each of which are running WSTP (wrapping the MPI program). I suppose the only benefit is that it moves the onus of talking to all work nodes to the intermediate note, which e.g. can be on their internal network. The user's remote kernel talks to only the intermediate node. $\endgroup$
    – Anti Earth
    Jan 16, 2019 at 23:41
  • $\begingroup$ @b3m2a1 Perhaps even the intermediate kernel has only one ObjectLink, but merely clones the TCP message it sent to a nominated work node, to all other nodes. I still really don't know if MPI will behave correctly in WSTP's seemingly async style $\endgroup$
    – Anti Earth
    Jan 16, 2019 at 23:43
  • $\begingroup$ "Where can an archive for Linux be downloaded?" You mean you do not have Mathematica for Linux at all, and you are attempting to connect from a Mac through WSTP? Perhaps you can contact Wolfram Research and ask if they can give you the Linux WSTP dev kit. I know that a separate license applies, so maybe they will oblige. It is not possible to mix Linux and Mac binaries. You absolutely need the Linux dev kit. $\endgroup$
    – Szabolcs
    Jan 17, 2019 at 7:18
  • $\begingroup$ (As for the MPI question, I do not have experience with MPI. Is there no main process in this system? I imagine the same problem would occur if you just want to save the computed results to a file! Only one thread should save.) $\endgroup$
    – Szabolcs
    Jan 17, 2019 at 7:23

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.