MathLink (WSTP) vs LibraryLink performance

Question

Has anyone benchmarked LibraryLink vs MathLink when passing large arrays?

Specifically I would like to know how performance compares when using the following to pass and return a large array:

• LibraryLink with {Real, _, Automatic} argument and return types
• LibraryLink with LinkObject argument and return types
• MathLink (WSTP) with MLPutRealArray() and MLGetRealArray() (or equivalent), through LinkProtocol -> "SharedMemory".

I suspect that someone has already done this benchmarking. If not, I'll self-answer in a few days.

Knowing how great the LibraryLink advantage is will help in deciding when it's worth it to sacrifice MathLink's flexibility for performance.

Answer 1

At the end of this post you'll find the code for a small benchmark to compare LibraryLink with standard passing vs LibraryLink with MathLink based passing on two counts:

• function call overhead; use a function that does nothing, has no arguments, returns nothing (llNone and mlNone for LibraryLink and MathLink, respectively)

• passing real arrays; use a function that takes a real array and returns it (llArr and mlArr)

I did not yet test installable MathLink programs running in a separate process. Below "MathLink" refers to a LibraryLink function using MathLink based argument passing and return. For LibraryLink I used Automatic passing ("Constant" should be faster).

The MathLink overhead is about 5.5 times greater than LibraryLink.

time[llNone[]]
(* 2.02255*10^-7 *)

time[mlNone[]]    
(* 1.09718*10^-6 *)

This shows the time it takes to cycle a packed real array through a function, as a function of array size. Blue is LibraryLink, yellow is MathLink.

For LibraryLink, the time to pass the array starts to dominate at an array size of about 500.

At this array size, LibraryLink is about 25 times faster than MathLink.

arr = RandomReal[1, 500];

time[llArr[arr]]
(* 7.78001*10^-7 *)

time[mlArr[arr]]
(* 0.0000202976 *)

When working with real arrays, using LibraryLink seems to be well worth it if performance is important.

---

Mathematica code

<< CCompilerDriver`

CreateLibrary[{"test.c"}, "testlib"]

llNone = LibraryFunctionLoad["testlib", "testcall1", {}, "Void"];

llArr = LibraryFunctionLoad["testlib", "testcall2", {{Real, _}}, {Real, _}];

mlNone = LibraryFunctionLoad["testlib", "testcall3", LinkObject, LinkObject];

mlArr = LibraryFunctionLoad["testlib", "testcall4", LinkObject, LinkObject];

SetAttributes[time, HoldAll]
time[expr_] :=
 Module[{t = 0., n = 1/2},
  While[t < 1.,
   n *= 2;
   t = First@AbsoluteTiming@Do[expr, {n}]
   ];
  t/n
  ]

libLinkRealArray = Table[
   {len, With[{arr = RandomReal[1, len]}, time[llArr[arr]]]},
   {len, 2^Range[20]}
   ];

mathLinkRealArray = Table[
   {len, With[{arr = RandomReal[1, len]}, time[mlArr[arr]]]},
   {len, 2^Range[20]}
   ];

ListLogLogPlot[{libLinkRealArray, mathLinkRealArray}, Axes -> False, 
 Frame -> True, FrameLabel -> {"array length", "time (s)"}]

C code

// test.c

#include "mathlink.h"
#include "WolframLibrary.h"

DLLEXPORT mint WolframLibrary_getVersion() {
    return WolframLibraryVersion;
}

DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) {
    return LIBRARY_NO_ERROR;
}

DLLEXPORT void WolframLibrary_uninitialize(WolframLibraryData libData) {
    return;
}

// LibraryLink, no argument
DLLEXPORT int testcall1(WolframLibraryData libData, mint Argc, MArgument * Args, MArgument Res) {
    return LIBRARY_NO_ERROR;
}

// LibraryLink, cycle a real array
DLLEXPORT int testcall2(WolframLibraryData libData, mint Argc, MArgument * Args, MArgument Res) {
    MTensor t1, t2 = NULL;
    t1 = MArgument_getMTensor(Args[0]);
    libData->MTensor_clone(t1, &t2);
    MArgument_setMTensor(Res, t2);
    return LIBRARY_NO_ERROR;
}

// MathLink, no arguments
DLLEXPORT int testcall3(WolframLibraryData libData, MLINK mlp) {
    return LIBRARY_NO_ERROR;
}

// MathLink, cycle a real array
DLLEXPORT int testcall4(WolframLibraryData libData, MLINK mlp) {
    double *data;
    int *dims;
    char **heads;
    int d;
    int argc;
    if (!MLTestHead(mlp, "List", &argc))
        return LIBRARY_FUNCTION_ERROR;
    if (argc != 1)
        return LIBRARY_FUNCTION_ERROR;
    if (!MLGetReal64Array(mlp, &data, &dims, &heads, &d))
        return LIBRARY_FUNCTION_ERROR;  
    MLNewPacket(mlp);
    MLPutReal64Array(mlp, data, dims, heads, d);
    MLReleaseReal64Array(mlp, data, dims, heads, d);
    return LIBRARY_NO_ERROR;
}