Mathematica Library Link : How to use non standard 'mint' e.g. 'unsigned int' or 'long long int'

Question

I use Mathematica 8.0.4 on Mac OSX 64 bit, and gcc as C compiler. I am trying to write a LibraryLink function that deals with large positive integers i.e. greater than $2^{31}-1$. By dealing with, I mean two different cases:

1. My function takes a large integer as input and/or returns a large integer as output

2. My function takes standard integers as input and output by produces intermediate large integers and therefore should probably declare and use unsigned int or long long int integer types.

How can I do this?

For 1. the available functions (MArgument_getInteger(Args[i]) and MArgument_setInteger(Res,i)) seem to handle 32 bit integers ranging from $-2^{31}$ to $2^{31}-1$ only (mint). Is there a way to transfer different types of integers?

For 2. I have tried to declare/use unsigned int or unsigned long or long long int local variables in the C functions, but either it does not compile or the declaration is somehow ignored and the integer is in fact considered a mint. In the WolframLibrary.h file, I tried to change the definition of mint from int to unsigned long for example, without success.

I would like to stick to the LibraryLink framework, because apart from this problem, I find it relatively predictable and easy to implement. I am aware of the MathLink functions MLGetInteger64() and MLPutInteger64(), but MathLink on my machine has proven desperately buggy. I copy-pasted entire examples from Minimal effort method for integrating C++ functions into Mathematica and associated conversations and often it just did not work. I thought that because MathLink is by nature asynchronous, it is very sensitive to low level communication protocols among processes forever out of control. Additionally, the reason I use LibrayLink is speed. MathLink brings an overhead that I suspect would slow down the process.

Answer 1

LibraryLink vs. MathLink

The Wolfram Library Link is a way to run code which is placed in a so called shared or dynamic library directly by loading the library into a running MathKernel. One way to send and to receive data are, as you mentioned, the MArgument_* functions.

But this is not the only way! As shown in the examples of the LibraryLink Tutorial you can easily send and retrieve any Mathematica expression by calling appropriate MathLink functions.

Therefore, let me first point out clearly, that LibraryLink is not the opposite of MathLink. MathLink is the name of the protocol (and the API to it) which is used to speak with a MathKernel. LibraryLink lets you load dll's and of course, you can use MathLink inside your dll's. What you may mean is, that MathLink was used to build executable programs which were then started from inside Mathematica and used. I let this count as being kind of the opposite of loading a dll (not really, but lets not be petty-minded here).

How to send/receive long with the LibraryLink

What you want to try is the following: Write a dll function like the one in the examples but instead of getting a string, you call for instance MLGetLongInteger or whatever you like

DLLEXPORT int reverseString( WolframLibraryData libData, MLINK mlp)
{
    int res = LIBRARY_FUNCTION_ERROR;
    long len;
    const char *inStr = NULL;
    char* outStr = NULL;

    if ( !MLCheckFunction( mlp, "List", &len)) 
        goto retPt;
    if ( len != 1) 
        goto retPt;

    if(! MLGetString(mlp, &inStr))
        goto retPt;

    if ( ! MLNewPacket(mlp) ) 
        goto retPt;

    outStr = reverseStringImpl(inStr);

    if (!MLPutString( mlp,outStr)) 
        goto retPt;
    res = 0;
retPt: 
    if ( inStr != NULL)
        MLReleaseString(mlp, inStr);
    if ( outStr != NULL)
        free( (void*) outStr);
    return res;
}

You have to send the result back through the MathLink! If you want to load this function into your running session, please note, that you have to give LinkObject as parameter- and return-type:

funReverse = 
  LibraryFunctionLoad["demo_mathlink", "reverseString", LinkObject, 
   LinkObject];

Debugging a MathLink-executable

This is pretty easy. Build your program with debugging symbols (-g switch) and then you start your MathLink-executable inside a debugger and set a break-points where you like.

Debugging a LibraryLink dll (can contain MathLink code!)

Say you want to debug the above function. An easy trick is the following. Write an infinite loop at the beginning like this

DLLEXPORT int reverseString( WolframLibraryData libData, MLINK mlp)
{

    int stopHere = 1;
    while(stopHere);

    int res = LIBRARY_FUNCTION_ERROR;
    long len;
    /* ... */

Then you load your dll-function like you would do in a normal run and call it from inside Mathematica. What now happens is clear: the call hangs in the loop and Mathematica displays a golden cell-bracket.

Now you start your debugger and you connect to your running MathKernel process. Here, in my Intel-debugger in Linux, this works like that (it should work in the same way with other debuggers too)

1. Click on the button Attach to process
2. Select the MathLink process or look-up the process-id and use this
3. Click Pause to pause the execution of the infinite loop
4. Set the variable "stopHere" to 0 with your debugger
5. Set breakpoints or go step-wise forward

Answer 2

For reference, I put my code lines and comments on the site. Nothing new for the experienced, I am certain, but maybe it can spare a few hours to a newcomer to Mathematica&Co, like me.

I have (1) followed Szabolcs advice and insisted with MathLink (the framework with the .tm file) and tried (2) Halirutan solution of using Mathlink commands in the Library Link framework.

Below are several ways to import/export large integers (and by extension many types of expressions) via (1) MathLink or (2) MathLink inside LibraryLink.

(My system: OSX 10.6.8, Mathematica 8.0.4, gcc 4.2.1 build 5666, TextWrangler)

1a/ MathLink .tm file

// Template
:Evaluate:      BeginPackage["MyPackage`"]
:Evaluate:      add4ML::usage = "Input=(32bit int, 64bit int). Output=64bit int."
:Evaluate:      add4ML::badargs = "Input=(32bit int, 64bit int)."
:Evaluate:      Begin["`Private`"]

:Begin:
:Function:      addInt
:Pattern:       add4ML[i_Integer, j_Integer]
:Arguments:     {i, j}
:ArgumentTypes: {Manual}
:ReturnType:    Manual
:End:

:Evaluate:      End[ ]
:Evaluate:      EndPackage[ ]

static long addIntImpl(int i, long j);
void addInt(void);

#include <stdio.h>
#include <stdlib.h>
#include "mathlink.h"

void addInt(void) {
    int i;
    long j, sum;
    MLGetInteger32(stdlink, &i);
    MLGetInteger64(stdlink, &j);
    sum = addIntImpl(i, j);
    MLPutInteger64(stdlink, sum);
}    

static long addIntImpl(int i, long j) {
    return i+j;
}

int main(int argc, char* argv[]) {
    return MLMain(argc, argv);
}

Note: The type of each input/output/local variable is explicit (MathLink for input/output and standard C for local variables) and you have full control, and no bad surprise. I use OSX so my main function is simple. I understand there is more to write for Windows.

To build and call the function

addIntexe = CreateExecutable[src4, "addInt", "Debug" -> True]
link4 = Install[addIntexe]

Note: For debugging, the simplest I found is this command. You can use it with LibraryLink too (see below). For most practical purposes it has the same effect as "ShellOutputFunction"->Print (http://stackoverflow.com/a/7772365/1420713).

2a/ MathLink inside LibraryLink (following Halirutan suggestion), importing/exporting individual integers

#include <stdlib.h>
#include "mathlink.h"
#include "WolframLibrary.h"

DLLEXPORT mint WolframLibrary_getVersion( ) {
    return WolframLibraryVersion;
}
DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) {
    return LIBRARY_NO_ERROR;
}
DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) {
return;
}

static long addIntImpl(int i, long j) {
    return i+j;
}

DLLEXPORT int addInt(WolframLibraryData libData, MLINK mlp)
{
    int i1;
    long li2, sum;
    long len;

    if (!MLCheckFunction(mlp, "List", &len)) 
        goto retPt;
    if (len != 2) 
        goto retPt;
    if (!MLGetInteger32(mlp, &i1)) 
        goto retPt;
    if (!MLGetInteger64(mlp, &li2)) 
        goto retPt;
    if (!MLNewPacket(mlp)) 
        goto retPt;    
    sum = addIntImpl(i1, li2);
    if (!MLPutInteger64(mlp,sum)) 
        goto retPt;
    return LIBRARY_NO_ERROR;
retPt: 
    return LIBRARY_FUNCTION_ERROR;
}    

To build (the same Debug option helps) and call the function

lib2 = CreateLibrary[src2, "addInt", "Debug" -> True]
add2 = LibraryFunctionLoad[lib2, "addInt", LinkObject, LinkObject]

Note: Like in the MathLink framework (1a/), you control every variable and import/export, so full control and no problem.

2b/ MathLink functions inside LibraryLink, importing a list of Integers (I don't repeat the #include and standard DLL)

static long sumListIntImpl(int *array, long len) {
    int i;
    long sum;
    sum = 0;
    for(i=0; i<len; i++) {
        sum += array[i];
    }
    return sum;
}    

//Code
DLLEXPORT int sumListInt(WolframLibraryData libData, MLINK mlp)
{
    int *array;
    long sum, len, c;
    if (!MLCheckFunction(mlp, "List", &len)) 
        goto retPt;
    if (len != 1) 
        goto retPt;
    if (!MLCheckFunction(mlp, "List", &len)) 
        goto retPt;
    if (len <= 0) 
        goto retPt;
    array = (int*) calloc(len, sizeof(int));
    for(c=0; c<len; c++) {
        if (!MLGetInteger32(mlp, &(array[c]))) 
        goto retPt;
    }
    if (!MLNewPacket(mlp)) 
        goto retPt;
    sum = sumListIntImpl(array, len);
    if (!MLPutInteger64(mlp,sum)) 
        goto retPt;
    return LIBRARY_NO_ERROR;
retPt: 
    return LIBRARY_FUNCTION_ERROR;
}

To build and call

lib3 = CreateLibrary[src3, "addInt", "Debug" -> True]
sum3 = LibraryFunctionLoad[lib3, "sumListInt", LinkObject, LinkObject]

Note: If you want to call your function with sum3[{1,3,7}], you need 2 "List" (my case), if you want to call sum3[1,3,7], you need only 1 "List".

2c/ the problematic case, ie the standard LibraryLink framework without the MathLink functions

static int multImpl(int i, int j) {
    return i*j;
}
DLLEXPORT int mult(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument     Res) {
    mint I0, I1, I2;
    long I3, I4;
    //int I3, I4;
    I0 = MArgument_getInteger(Args[0]);
    I1 = MArgument_getInteger(Args[1]);
    I3 = I0+2147483647;
    if(I3<0) {I4 = 0;} else {I4 = I3-2147483647;}
        I0 = I4;
    I2 = multImpl(I0, I1);
    MArgument_setInteger(Res, I2);
    return LIBRARY_NO_ERROR;
}

To build and call

lib1 = CreateLibrary[src1, "mult", "Debug" -> True]
mult1 = LibraryFunctionLoad[lib1, "mult", {Integer, Integer}, Integer]

Note: Here, as expected the MArgument_set/get* functions do now allow to pass large integers (>2^31-1). But I would have expected that locally, I could have declared long integers and been able to run calculations involving large integers, but the function above returns zero whether I declare I3 and I4 as long or int.

Conclusion: To put it short, in my own (short) experience with Mathematica, when dealing with MathLink and/or LibraryLink, when in doubt go manual (ie use MathLink functions with explicit types, a bit verbose but precise) to pass around the parameters, or stay strictly within the boundaries documented (ie MArgument_* functions and mint/double/mcomplex/etc types for LibraryLink, and Integer/Real/IntegerList/RealList/etc predefined argument types for MathLink).