# Minimal effort method for integrating C++ functions into Mathematica

As of Mathematica 8, what is the minimal effort way to integrate an existing C++ function into Mathematica?

I think we have these:

• MathLink (it was quite long ago I used it last time)
• communication through pipes/files (Import is slow, ReadList not so much)
• Mathematica 8's C code generation features (??) (apparently this is not relevant)

The keyword here is minimal effort. Which is the most convenient way, including the learning curve for the particular method?

I'm mainly interested in doing it on Windows. My particular function computes a long list of real numbers. The function input consists only of a few scalars (int & double).

An answer of the type "You'll likely spend the least time if you use technology X" is useful. A concrete example of how to do it using a small function is even better.

Each and every answer I got is great, and each demonstrates a different technology. It's impossible to choose a definitive one. Here's a small "table of contents" for them:

A presentation about the topic from the Wolfram Technology Conference is here (CDF file).

## Experiences

I used two of these methods: the .NETLink method and Library Link. These are my personal experiences:

• .NETLink This was easy to set up, however, I'd recommend it only for when you already have a compiled DLL from which you need to load functions (e.g. do something like this). If you compile your own DLL: once the DLL is loaded, it is locked and cannot be overwritten. A quick and dirty way to allow overwriting it is realoding NETLink with ReinstallNET[].

Advantages: Very quick and easy, only Mathematica code is needed if the functions are already compiled into a DLL.

Disadvantages: Windows-only, does not parallelize from within Mathematica, and the calculation will not be interruptible.

• Library Link It's much easier to use this than what it looks like at first. Less setup is needed than in the case of MathLink, and compilation is automated from within Mathematica.

Advantages: Also quite easy, but both Mathematica and C code are needed. It is parallelizable, and easily made interruptible. The library can be unloaded (for recompilation) using LibraryUnload.

Update: While at the time of writing the question, loading DLLs with .NET/Link seemed easier, now I'm commonly using LibraryLink, which I recommend to anyone with a similar problem!

-

## migrated from stackoverflow.comJul 18 '12 at 11:42

This question came from our site for professional and enthusiast programmers.

Is that a List of scalars, or a finite number of parameters? –  rcollyer Nov 15 '11 at 18:04
@rcollyer By "sclaras" I meant that I have a finite number of parameters, each of which is a number. I wanted to point out that I don't need to pass variable (or fixed...) length vectors/list –  Szabolcs Nov 15 '11 at 18:06
That's what I thought you meant, I was just trying to be clear. –  rcollyer Nov 15 '11 at 18:11
CCompilerDriver/tutorial/Compilation#472267182 in the Mathematica documentation is probably a good starting point. –  Brett Champion Nov 15 '11 at 18:22
The C code generation features are about converting Compile expressions to C code for speed or use them outside Mathematica, rather than accessing existing C code from the kernel. –  Joel Klein Nov 16 '11 at 15:45

On Windows, C/C++ functions that have been compiled into DLLs can be accessed reasonably easily using NETLink. Let's say we have the following C++ DLL definition:

#include "stdafx.h"

BOOL APIENTRY DllMain(HMODULE module, DWORD reason, LPVOID reserved) {
return TRUE;
}

extern "C" __declspec(dllexport)
void helloMma(double a, double b, int n, double m[]) {
for (int i = 0; i < n ; ++i) {
m[i] = a * i + b;
}
}


We can import this function into Mathematica as follows:

Needs["NETLink"]

$dllPath = "C:\\some\\path\\to\\hellomma.dll"; helloMma = DefineDLLFunction[ "helloMma",$dllPath, "void", {"double", "double", "int", "double[]"}
]


Alas the calling sequence for this function is complicated slightly by the fact that it returns its result by destructively overwriting a statically allocated array of doubles (not an uncommon occurrence). To do this from Mathematica, we must pre-allocate the result vector using NETNew:

In[23]:= NETBlock@Module[{n, result}
, n = 10
; result = NETNew["System.Double[]", n]
; helloMma[3, 5, n, result]
; NETObjectToExpression[result]
]
Out[23]= {5., 8., 11., 14., 17., 20., 23., 26., 29., 32.}


Note that all the usual Bad Things would happen if the pre-allocated buffer were overrun. NETBlock is used to ensure that the storage allocated for the buffer is released when we are done with it.

I will point out a couple of "gotchas". Make sure that the DLL is compiled as 32-bit or 64-bit to match the version of Mathematica that you are running. Also, note that the exported DLL function in the example is declared as extern "C". This prevents the C++ compiler from "mangling" the name and makes it easier to reference in the DefineDLLFunction declaration (in this case the Visual Studio C++ mangled name was ?helloMma@@YAXNNHQEAN@Z).

-
I'm accepting this as a minimal effort solution that worked for me on Windows, but all the answers were great and very useful! –  Szabolcs Nov 25 '11 at 16:19
Is it possible to unload the DLL so I can recompile the DLL without quitting the kernel? –  Szabolcs Nov 25 '11 at 16:28
Unloading the DLL might be tricky. If Mma is using .NET interop to load the DLL, then unloading might not be practical -- one would have to load the DLL into a separate application domain and unload that domain. I'm not sure whether Mma would give us control over application domains. If Mma is not using .NET interop, then calls to the standard Win32 API functions GetModuleHandle and FreeLibrary through NETLink might do the trick. This is all speculation on my part at this point -- I have not tried either approach myself. –  WReach Nov 25 '11 at 17:54
For practical purposes, ReinstallNET[] works great for "unloading". Of course it destroys everything else .NET related too, but all I wanted was to recompile my DLL without quitting the Mathematica kernel (and I wasn't using .NET for anything else). I should update the post eventually with my experiences with this method and Library Link. The two significant disdvantages of the .NETLink method that remain are that 1. it's Windows only 2. it's not parallelizable from within Mathematica (think ParallelTable). Correct me if I am wrong. –  Szabolcs Dec 21 '11 at 17:15
@Szabolcs, in the above exapmle, the use of NETBolck would allow you to recompile your dll file without UninstallNET[] or ReinstallNET[] –  unstable May 18 '12 at 11:37

Here is my pitch to use LibraryLink, which is a really nice new technology in version 8. I am not going to pretend this is easy by any stretch of the imagination, because it involves a decent amount of knowledge of both Mathematica and C compilers. In this particular case I am using Visual Studio C++ Express plus the Microsoft Windows SDK 7.1. For the record, I had quite a bit of help from Joel Klein with this answer.

LibraryLink is set up to find most compilers on your system, but in my case I had to restart after installing the above tools (although looking back I think restarting my open frontend might have done the trick as well).

There are several examples of how to use LibraryLink in the documentation, but this example I wrote from scratch. All LibraryLink C code is compiled with the CreateLibrary function (which is located in the CCompilerDriver package.

Mathematica side

I'll skip the source for now, and focus on the Mathematica commands. First we loads the package to run the library link utility functions:

Needs["CCompilerDriver"]


Next, we load the source file and create a library (.dll) from it:

myLibrary = CreateLibrary[
{"c:\\users\\arnoudb\\myLibrary.c"},
"myLibrary",
"Debug" -> False];


Next, we load the one function defined in this library:

myFunction = LibraryFunctionLoad[
myLibrary,
"myFunction",
{{Real, 2}}, {Real, 2}];


Next we use this function:

myFunction[{{1, 2, 3, 4}, {5, 6, 7, 8}}]


which returns the square of every matrix entry: {{1., 4., 9., 16.}, {25., 36., 49., 64.}}.

Finally, we unload the library (needed if we make changes to the source file and reload everything above):

LibraryUnload[myLibrary];


Full inputs are here.

C side

There is a lot of boiler plate magic, to make things 'work'. These first four line need to always be included:

#include "WolframLibrary.h"
DLLEXPORT mint WolframLibrary_getVersion(){return WolframLibraryVersion;}
DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) {return 0;}
DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) {}


This is the actual function that you write. The function header is always the same, you get the actual function arguments from the MArgument_* api functions:

DLLEXPORT int myFunction(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){

int err; // error code

MTensor m1; // input tensor
MTensor m2; // output tensor

mint const* dims; // dimensions of the tensor

double* data1; // actual data of the input tensor
double* data2; // data for the output tensor

mint i; // bean counters
mint j;


This gets the input tensor and dimensions and sets up the output tensor and actual data:

 m1 = MArgument_getMTensor(Args[0]);
dims = libData->MTensor_getDimensions(m1);
err = libData->MTensor_new(MType_Real, 2, dims,&m2);
data1 = libData->MTensor_getRealData(m1);
data2 = libData->MTensor_getRealData(m2);


The actual interesting stuff, this squares each element:

 for(i=0;i<dims[0];i++) {
for(j=0;j<dims[1];j++) {
data2[i*dims[1]+j] = data1[i*dims[1]+j]*data1[i*dims[1]+j];
}
}


This set the return value (and yes, you don't want to return the 'err' value here):

 MArgument_setMTensor(Res, m2);
return LIBRARY_NO_ERROR;
}


The full code is here.

-
For me, the case was made with the function CreateLibrary, +1. –  rcollyer Nov 16 '11 at 3:14
There are plenty of examples in the documentation, I guess it's easier to learn by examples, I tested a dummy example once in VisualC++ and it worked (don't forget "extern C" sometimes) here are some links: reference.wolfram.com/mathematica/LibraryLink/tutorial/… reference.wolfram.com/mathematica/TetGenLink/tutorial/… reference.wolfram.com/mathematica/LibraryLink/tutorial/… reference.wolfram.com/mathematica/CCodeGenerator/tutorial/… –  Faysal Aberkane Nov 16 '11 at 21:54
It looks like I'll go with this approach after all! But there's a little problem in the code: after macro expansion, MArgument_setMTensor(Res, m2) will look like (*(Res.tensor)) = m2, i.e. err = MArgument_setMTensor(Res, m2); will eventually "evaluate" to err = m2, which is incorrect and will also prevent the code from being compiled as C++. Could you correct this? Also, if I got it right, one must add EXTERN_C in front of all the DLLEXPORTs if the code is C++. –  Szabolcs Nov 27 '11 at 12:16
Another important point when using C++ with the MinGW compiler: it's necessary to pass "Libraries" -> "-lstdc++" to CreateLibrary[], otherwise there will be linking errors for any non-trivial C++ program. Is this a compiler driver bug by any chance? I would have expected this to happen automatically when using "Language" -> "C++" (since the "Libraries" option will be different for different compilers -- so the Mathematica side of my code is now not completely portable) –  Szabolcs Nov 27 '11 at 13:39

Presuming that your c++ code is already written, then I don't know how the code generation feature would be helpful. That said, in order of simplicity, I would have Get, ReadList, Import, and both LibraryLink and MathLink.

Get and ReadList are by far the simplest. Provided that your c++ program outputs to stdout (std::cout), then it is simply

val = (<<"!command")


or,

ReadList["!command", Number (*or some other specifier*)]


If you need to control your external program more directly, or pass additional data to it (like run your function multiple times), then this method is more difficult and may require you to open pipes to communicate through (1, 2).

Import would require you to conform to some data format, which while very doable, is not as simple as just putting numbers onto stdout.

I have never used LibraryLink, but a quick perusal of the docs implies that it is a simplification of the MathLink interface. More specifically, it is easier to create a function that can communicate with Mathematica, but the intricacies of sending data back and forth remain. To return a List, though, is not to bad, and may well be worth it. The catch, of course, is that you need to provide a c interface to your function, which differs from the MathLink form, but is still necessary. So, I'm not sure there is an advantage here for one or the other.

-
To the downvoter, I am interested in what was missing from my answer. Did I give to little detail? Or, was it considered incorrect in some way? Either way, would you please elaborate. I'm not trying to complain here, I'm just curious what I could have done instead. –  rcollyer Nov 15 '11 at 21:31
It's possibly from all the pedestrian traffic in the C++ tag. Surely not a downvote from a Mathematica user. An upvote from me to get you closer to a silver :) –  yoda Nov 16 '11 at 14:44
@yoda, you're probably right. They, most likely, got "mad" when I didn't post much code. Oh well. Any way, thanks for the upvote, although I'm still over a hundred shy of the silver. And, I've got to run over Heike first ... :D –  rcollyer Nov 16 '11 at 14:57
This sort of downvotes seems to just be a fact of life, a drive-by shooting of sorts. I personally think that it is a moral obligation of the downvoter to explain the reasons (I don't downvote myself), but obviously not everyone thinks the same. In any case, here is another one towards your silver - +1 :) –  Leonid Shifrin Nov 16 '11 at 15:41
@LeonidShifrin, belisarius and I had a long conversation about downvotes and associated messages. For the most part, he doesn't think the messages influence the people who need it, and the people who don't, aren't likely to be downvoted. In this case, I was honestly curious what they figured was needed, especially considering I do pay attention. But, alas, it was not to be. :P (I downvote rarely. Reserving it for incorrect, or misleading, answers. And, usually, it is outside of mathematica.) –  rcollyer Nov 16 '11 at 15:56

I suggest to use MathLink, which you can automate using the CCompilerDriver. This is a safe alternative, since you won't crash the kernel if your code crashes. Once tested, this should not be hard to convert to library link. As an explicit example, consider a function which receives a list of integers and squares it. First, here is a function to create the boilerplate code:

makeMLinkCodeF =
StringJoin[
"

#if __BORLANDC__
#pragma argsused
#endif

int PASCAL WinMain( HINSTANCE hinstCurrent, HINSTANCE
\ hinstPrevious, LPSTR lpszCmdLine, int nCmdShow)
{
\tchar  buff[512];
\tchar FAR * buff_start = buff;
\tchar FAR * argv[32];
\tchar FAR * FAR * argv_end = argv + 32;

\thinstPrevious = hinstPrevious; /* suppress warning */

\tif( !MLInitializeIcon( hinstCurrent, nCmdShow)) return 1;
\tMLScanString( argv, &argv_end, &lpszCmdLine, &buff_start);
\treturn MLMain( (int)(argv_end - argv), argv);
}

#else

int main(int argc, char* argv[])
{
\treturn MLMain(argc, argv);
}

#endif"] &;


It is pretty ugly, but you only need to define it once. Perhaps, a better way would be to rewrite it using Symbolic C, but I had no time to do it. Now, here is the code for our function:

code =
"
extern void squareList(int * data, long len);

void squareList(int * data, long len){
int *d = data;
while(d-data<len){
d++;
}
}
";


And here is the template:

template =
StringReplace[#,"\n"~~Whitespace:>"\n"]&@
"
void squareList P((int *, long));

:Begin:
:Function:       squareList
:Pattern:        squareList[data_List]
:Arguments:      { data }
:ArgumentTypes:  { IntegerList }
:ReturnType:     Manual
:End:

";


(StringReplace was needed since otherwise the template would not copy-paste correctly from SO - it would contain spaces on the left, which would prevent us from creating an executable. You don't have to do this in the interactive session).

We must now load the compiler driver:

Needs["CCompilerDriver"];


The following code prepares our stuff:

fullCCode = makeMLinkCodeF[code];
projectDir = "C:\\Temp\\MLProject";
If[! FileExistsQ[projectDir], CreateDirectory[projectDir]]
pname = "squareList";
files =
Export[FileNameJoin[{projectDir, pname <> #2}], #1, "String"] &,
{{fullCCode, template}, {".c", ".tm"}}];


Now we attempt to create an executable:

In[19]:= exe=CreateExecutable[files,pname]
Out[19]= C:\Users\Archie\AppData\Roaming\Mathematica\SystemFiles\LibraryResources
\Windows-x86-64\squareList.exe


We now install it:

In[20]:= Install[exe]
LibraryResources\Windows-x86-64\squareList.exe",10,8]


And finally use it:

In[21]:= squareList[Range[5]]
Out[21]= {1,4,9,16,25}

In[23]:= Uninstall[exe]
Out[23]= C:\Users\Archie\AppData\Roaming\Mathematica\SystemFiles\LibraryResources
\Windows-x86-64\squareList.exe


There were a number of steps here, but all of them are within Mathematica, and most of them are common for all functions and can be factored away. If you end up doing lots of coding, I'd create a Mathematica-based DSL which would generate Symbolic C for your functions and also your templates. If / when time permits, I intend to improve this answer by automating the procedure more, and by replacing the fragile string-based pieces with functions to generate them.

-
Would be very interesting to see the automation stuff :). –  nixeagle Dec 19 '11 at 22:39
@nixeagle Yes, I also would like to do this, since the string-based manipulations are fragile. This is not really difficult, just need some free time :). –  Leonid Shifrin Dec 20 '11 at 16:41

You may want to look at this presentation: Integrating C and Mathematica.

In the past, I have found using .NET/Link to be the easiest. You can call C DLL's very easily on Windows, without the need for templates as in MathLink.

-