Portable makefile for wscc builds

What's the canonical way to write a portable makefile for wscc programs? The makefile typically requires the location of the wscc script and wstp.h header file.

I see in this example they are hardcoded:

WSTPLINKDIR = /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Links/WSTP/DeveloperKit
SYS = Linux  # Set this value with the result of evaluating $SystemID CADDSDIR =${WSTPLINKDIR}/${SYS}/CompilerAdditions  Can we do something better? For ordinary linux packages, I'd use e.g.,pkg-config. Do I just have to tell anyone who wants to build my code to search for the correct directories (possibly with locate) and hack the makefile themselves? Is there a way within a Mathematica shell to get the location of them reliably? Even GetEnvironment[{"MATHEMATICA_HOME"}] isn't enough. • I added an example to the answer. – Szabolcs Aug 28 '17 at 10:29 • I looked at your code on GitHub, and it looks like this could also be done with LibraryLink instead of MathLink. MathLink is easier to set up with its .tm files, but LibraryLink is going to be faster (important if your functions is called in a loop) and offers more features (it includes MathLink as a subset). I have a package, LTemplate, which makes LibraryLink just as easy to work with as MathLink/.tm files. – Szabolcs Aug 28 '17 at 11:17 2 Answers There is no platform-independent way to detect the location of Mathematica from a terminal shell or Makefile. You also have to decide which version to use when multiple versions are installed (a fairly common situation). For a few platform-dependent attempts to detect locations, take a look at the makefiles we used for MATLink. None of these ways are robust though. However, if you are creating a Mathematica package with some binary components, and all the C source code is contained within the package (without hard-to-control external dependencies, such as MATLAB was for MATLink), then you can compile from within Mathematica. The installation directory is pointed to by $InstallationDirectory. All the relevant paths (such as the MathLink libraries) are at a fixed location relative to that.

The CCompilerDriver package seems to have much of this automated. It takes care not only of the location of MathLink libraries, but also most platform-specific or compiler-specific details (such as different syntax for gcc and MSVC). See the "WSTP Executables" section in its documentation. Note that you must use MathLink instead of WSTP if you are working with CCompilerDriver.

A nice solution is to set up your package to auto-compile its binary components on first load. If you do this, keep in mind that you can only count on your users having a C compiler installed on Linux. On OS X and Windows it is better to ship pre-compiled executables.

Skeleton project that auto-compiles its MathLink component

The following is a simple proof-of-concept of a package that auto-compiles its MathLink dependency using CreateExecutable. For simple projects, it is sufficient to use CreateExecutable, and you do not need a Makefile. The advantage is that it will take care of most OS-specific or compiler-specific details, so you will not need to have separate makefiles for different operating systems / compilers.

To set up the project, create the following directory structure

AddTwo
|
|- Kernel
|    |
|    \- init.m
|
|
|- Sources
|     |
|     |
|
|
\


Here addtwoprog must be an empty directory. This is where the MathLink executables will be saved.

The parent of the top-level AddTwo directory (not AddTwo itself!) must be in the $Path. addtwo.tm and addtwo.c must be copied from the location opened by SystemOpen@ FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links",
"MathLink", "DeveloperKit", $SystemID, "MathLinkExamples"}]  If this location contains addtwo.cxx instead of addtwo.c, simply rename it to addtwo.c after copying. This is the contents of the other files: (* init.m *) Get["AddTwoAddTwo"]  (* AddTwo.m *) BeginPackage["AddTwo"] (* CCompilerDriver is not included in BeginPackage to avoid keeping it in the$ContextPath *)
Needs["CCompilerDriver"]

to ensure it is created in the correct context. *)
AddTwo::usage = "AddTwo[m, n] returns the sum of integers m and n.";

Begin["Private"]

$packageDir = DirectoryName[$InputFileName];
$executable = FileNameJoin[{$packageDir, "addtwoprog"}]; (* See Install[] documentation, Details section, for how it resolves directories into executables *)

(* This is the list of source files to be compiled. There may be multiple .c and .tm files,
all of which will be linked together. See the CreateExecutable[] documentation,
Details section, on how to include additional object files. See CreateObjectFile[]
on how to create an object file without also linking it into an executable. *)
$sourceFiles = FileNameJoin[{$packageDir, "Sources", #}]& /@ {"addtwo.tm", "addtwo.c"};

(* Just a colourful Print alternative *)
print[args___] := Print @@ (Style[#, Red]&) /@ {args}

If[
Install[$executable] ===$Failed
,
CreateExecutable[$sourceFiles, "addtwoprog", "TargetDirectory" -> FileNameJoin[{$executable, $SystemID}], (* the following two options are here to ease debugging *) "ShellCommandFunction" -> Print, (* see compilation command *) "ShellOutputFunction" -> Print (* see compiler messages *) ]; If[ Install[$executable] === $Failed , print["Failed to compile or run AddTwo executable. \[SadSmiley]"], print["Compiling successful! \[HappySmiley]"] ] ] End[] (* Private *) EndPackage[]  This package will auto-compile its C component when it is loaded for the first time. Try it with <<AddTwo ?AddTwo AddTwo[3,4]  The compiled binary will be stored in the package directory. On a second load of the package, it won't be re-compiled again. The best approach is probably to distribute these binaries to your users. This is especially important on OS X and Windows where you can't count on a compiler being already set up. Luckily, on these platforms it is easier to ensure that your binaries will work for everyone (e.g. see here for OS X). This skeleton package was tested on OS X / M11.1 and Linux / Raspberry Pi / M11.0. • Thanks. In mathlink, I found the bash attempts to do it: github.com/rsmenon/MATLink/blob/develop/MATLink/Engine/src/… – innisfree Aug 28 '17 at 9:13 • @innisfree It's MATLink. MathLink is the old name of WSTP. What you are linking to is Linux-specific, and doesn't even work across all Linux systems. (What if it's called wolfram instead of math? What if Mathematica is not in the PATH? What if multiple versions are in the PATH under different names?) It's the best we could do, and the installation instructions of MATLink tell people to edit the Makefile manually when needed. It's the nature of Linux that it's hard to make thing robust and foolproof ... but for the same reason it's not unreasonable to expect ... – Szabolcs Aug 28 '17 at 9:17 • @innisfree ... Linux users to do a bit more work to set up a package than Windows or Mac users. On Windows and Mac we provide precompiled binaries for MATLink (which is much easier to do on those platforms than on Linux). – Szabolcs Aug 28 '17 at 9:18 • @innisfree Actually if yours is purely a Mathematica package, you can just have it compile automatically when the package is loaded as Needs["YourPackage"]. Whether this is feasible depends on whether you have any hard-to-configure external dependencies (we had MATLAB for MATLink). If it's only your code, just compile from within Mathematica, and you'll be able to easily detect paths with $InstallationDirectory. – Szabolcs Aug 28 '17 at 9:20
• Thanks. sorry MATLink! Why doesn't a default linux mathematica install put these commands/header files in the usr/bin & usr/include already? wouldn't that be the best way? – innisfree Aug 28 '17 at 9:32

The answer to the narrow question of how to get the location of the headers from within Mathematica is to evaluate

FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links", "WSTP", "DeveloperKit",$SystemID, "CompilerAdditions"}]


has long given the location of the headers and libraries on Posix, and does so on all platforms starting from 11.0 (in MathLink, Windows was the odd man out until 11.2). Assuming 'math' is on the PATH, which it is by default on Linux, you could get this value on the command-line using

echo 'FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links", "WSTP", "DeveloperKit",$SystemID, "CompilerAdditions"}]' | math -batchoutput


Longer term, we plan to provide tools to integrate development into a cmake project.

• "has long given the location of the headers and libraries on Posix, and will do so on all platforms starting in 11.2." Does it not work on Windows? If not, why? – Szabolcs Aug 28 '17 at 16:35
• @Szabolcs There was an issue on Windows, but I realize now that I need to update versions in my answer. For WSTP, we made this change in 11.0, and it just now in 11.2 and we're making MathLink behave the same way. For older versions, there was an extra level between CompilerAdditions and its contents on Windows, because historically we provided several compiler-specific version (a VS version, a cygwin version, etc.). Also, Windows had a wierd Unix-style bin/lib/include hierarachy, so CompilerAdditions\mldev64 didn't actually include the libraries or headers. – Itai Seggev Aug 28 '17 at 17:08
• Nice, I will use DIR = \$(shell echo 'FileNameJoin[{$$InstallationDirectory, "SystemFiles", "Links", "WSTP", "DeveloperKit",$$SystemID, "CompilerAdditions"}]' | math -batchoutput | tr -d '"')` – innisfree Aug 29 '17 at 2:06