How to create dll so they can be LibraryFunctionLoad'ed by earlier versions of Mathematica

Question

I found that LibraryFunctionLoad fails to load functions from dylibss created by later versions of Mathematica.

When creating the library in more recent versions of Mathematica, how can I modify the source .cc file in or set certain options in CreateLibrary so that functions can be loaded by older versions of Mathematica?

Answer 1

I did not originally answer this because I do not have a full understanding of the situation. But partial information is better than no information, so here it goes:

In my experience, generally, LibraryLink libraries compiled with one version of Mathematica will also work with other versions. Below I will list a few exceptional situations when this is not the case.

But first, let us look at what it really means for a shared library to be created by a certain version of Mathematica. CreateLibrary will just call the compiler with certain options. You can see then using "ShellCommandFunction" -> Print. You can even create a LibraryLink library without a full Mathematica installation. The only things you need are:

• LibraryLink headers
• MathLink headers and library (binary), in case you are using MathLink functions in your project

Additionally, linking against different versions of a library that the Mathematica kernel also links against may cause trouble (and therefore limit compatibility between versions). An example would be Microsoft's C Runtime Library on Windows. My knowledge in this area is extremely lacking and I cannot tell you what exactly will cause compatibility problems, and whether it is okay to use a Visual C++ version that is different from what Mathematica itself was compiled with. All I can say that I have not yet run into trouble when switching MSVC versions—but that doesn't guarantee that no trouble is lurking beyond the next corner.

What about the header files?

The most relevant difference in the header files is the value of WolframLibraryVersion, which you are supposed to return from WolframLibrary_getVersion(). This is 3 between 10.0-11.1 and 4 in 11.2. This means that libraries created with 11.2 can't be loaded in 11.1 and earlier. However, libraries created in 11.1 do work in 11.2. I believe that when the kernel loads a library, it invokes WolframLibrary_getVersion(), and will pass a libData struct to the library according to the result. 11.2 has separate libData structs for v3 and v4 library interface versions, and will pass in the one compatible with your library.

There are other differences in the headers between 10.0-11.1, but these don't affect the loadability of the shared libraries. For example, only 10.4 and later support MRawArray. The reason why a library that uses raw arrays won't be loadable in 10.3 is not shared library compatibility, but the simple fact that LibraryFunctionLoad won't recognize "RawArray" as a type specification.

What about the MathLink libraries?

On OS X only, there was a compatibility break between 10.3 and 10.4 due to Apple's switch from libstdc++ to libc++. Mathematica 10.4 and later are compiled with libc++, and I believe they can only load the MathLink shared library compiled with the same. This means that on OS X you need to link to the appropriate version of the MathLink framework for your given version of Mathematica, which is 4.36 for M >= 10.4 (with libc++) and 4.25 for M <= 10.3 (with libstdc++).

In practice, a LibraryLink library compiled with 10.4 or later works only with 10.4 or later on OS X. One compiled with 10.3 or earlier works only in 10.3 or earlier.

However, since MathLink 4.25 does ship with M10.4+, it is possible to build a 10.3- compatible library with these versions. Itai Seggev shows how to do it on Wolfram Community:

In the compiler command line, replace

-F"/Applications/Mathematica 10.4.app/Contents/SystemFiles/Links/MathLink/DeveloperKit/MacOSX-x86-64/CompilerAdditions" -framework "mathlink"

with

-Wl,"/Applications/Mathematica 10.4.app/Contents/SystemFiles/Links/MathLink/DeveloperKit/MacOSX-x86-64/CompilerAdditions/mathlink.framework/Versions/4.25/mathlink"

(correcting the paths for your installation of Mathematica).

Other things to note:

• This problem does not affect installable MathLink programs, which run as a separate process. One compiled with an older MathLink library can still connect to a newer Mathematica kernel.

• Even if you compile for 10.3 or earlier, you can use link your shared library against libc++. It won't cause a conflict when you also link against a libstdc++-based MathLink library, and it is in fact necessary if you want to use C++11 features on OS X (also pass -mmacosx-version-min=10.9).

Once again I want to emphasize that the problems described in this section apply to OS X only.

Summary

In general, LibraryLink libraries compatible with one version of Mathematica will load in earlier or later versions as well, except:

• When there was a change in WolframLibraryVersion. This happened between 11.1 and 11.2. 11.2 can load older libraries, but 11.1 cannot load newer ones.

• On OS X only, there was a compatibility break between 10.3 and 10.4. You'll need a separate shared library for 10.0-10.3 and for 10.4-11.2.

My IGraph/M package relies on LibraryLink and aims to support Mathematica 10.0 and later on multiple platforms. So far I got away with providing a single binary on Windows (for 10.0+), a single one on Linux (for 10.0+) and two different ones on OS X (for 10.0-10.3 and 10.4, respectively).

---

Any corrections to the information above will be most welcome.