Cross-version compatibility of LibraryLink libraries

Question

I am looking for practical advice on the cross-version compatibility of LibraryLink libraries.

In particular, I would like to understand the significance of the WolframLibraryVersion define, which is normally returned from the WolframLibrary_getVersion function manually (meaning that one can in principle return whatever).

Under what circumstances will libraries compiled with an old version of Mathematica fail to load in a new version?

When will libraries compiled with a new Mathematica fail to load in an old Mathematica?

Does Wolfram provide any guarantees for binary compatibility? What about source compatibility?

Why is there a version value for WolframLibrary.h, but not for any of the ancillary headers like WolframSparseLibrary.h? Note that WolframLibraryVersion was not increased when new ancillary headers were added.

Particular concerns:

• Changes in WolframLibraryVersion (e.g. it changed from 3 to 4 between 11.1 and 11.2)

• New features introduced (e.g. the WolframRawArrayLibrary.h was introduced in 10.4)

• Changes in external dependencies (e.g. on OS X, M10.4 uses libc++, while previous versions used the incompatible libstdc++)

Answer 1

We cannot guarantee binary compatibility across versions, especially if using C++. Given that MSFT didn't even attempt to have cross-version compatibility across Visual C++ version until VS2017 (with a special pragma to refuse to even try to link against different versions), there isn't much we can do about it. We do make every effort to maintain that compatibility.

Generally speaking, a new version of Mathematica can load old versions unless there is a C++ runtime incompatibility. Old version can load newer version as long as their is neither a C++ rumtime incompatibility or a source-level incompatibility (e.g., it is using a new interface). I'm not knowledgeble enough about the guts of LibraryLink to tell you why the library version was bumped or its significance--I would have that it is the API level. I'll ask some folk to take a look.

For the libstdc++ vs libc++ issue specifically, the resolution is easy. Build the library twice, once against libstdc++ and libc++, then load the correct version based on $VersionNumber at runtime. This is what we did for example with GitLink, so that can serve as a useful reference.

mlobjfile = FileNameJoin[{$InstallationDirectory,
    "SystemFiles/Links/MathLink/DeveloperKit/MacOSX-x86-64",
    "CompilerAdditions/mathlink.framework/Versions/4.25/mathlink"}];
compileOpts = StringReplace[compileOpts, "10.9"->"10.7"] <> " -Wl,\"" <> mlobjfile <> "\"";

This links against the the libstdc++-based version of MathLink, and it changes the -mmacosx-version-min from 10.9 to 10.7, which effectively changes the C++ runtime to link against.

This in principle generalizes as long as you keep your old M-'s around. Build duplicate libraries in older versions, copy them under a new name into your application, and load the correct one based on versions. You could theoretically do this with API-level incompatibilities using appropriate #ifdefs, but I this isn't something I've seen done.

In version 11.2, we (finally) transitioned away from VS2013 to VS2017. So you may find you need to build a new version of the shared library for Windows. In this case, you may need both an old M- and an old VS to do the build.