# How to prevent the kernel from exiting when a LibraryLink function segfaults?

I do realize that this might be controversial, but I believe I have a real need for it this time.

If a process tries to access memory at a location it doesn't own, it will by default exit with a "segmentation fault" error. LibraryLink functions run in the kernel process, so if they crash, the whole kernel crashes. Mathematica is normally used interactively, so a kernel crash can be very unpleasant and lead to lost data ... (even if the front end doesn't crash and the notebook isn't lost).

How can I install some sort of protection so that when my LibraryLink function crashes, the kernel would print a warning about potential inconsistent state, but instead of exiting right away, attempt to give the user an opportunity to save (Export) their work?

Why do I believe that this is possible? Because MATLAB does it and sometimes R does it. Here's the dialog that MATLAB shows when a MEX function (equivalent of LibraryLink) crashes. Notice the Attempt to Continue button.

Why do I want it? I understand that the right solution is to just fix the crash. Or if that's really out of my control, then use MathLink and make it run in a separate process. But consider a situation like that of IGraph/M. It's a big LibraryLink project that can't be easily transferred to MathLink. It's using a large and complex library I didn't write myself that can crash in unexpected situations. That small window of opportunity for the user to save their work (when the kernel state is not completely corrupted) would really be a big improvement. igraph is not particularly crashy, but it does crash more often than builtin Mathematica functions.

What about catching SIGSEGV? My understanding is that on Unix systems it's possible to install a handler for SIGSEGV. Is this the right way to go? I don't want to catch all crashes, only those ones that are localized to my library. Is this possible? Doing a global modification to the kernel process seems like a bad idea.

Any solutions are going to be system-specific. I'm primarily interested in OS X solutions (but also Linux and Windows).

Cross posted on Wolfram Community

• It's completely possible that I am misunderstanding something and this is not possible (or really not a good idea) after all. I don't know much about processes, signals, etc. Any comments will be most appreciated! – Szabolcs Oct 14 '15 at 8:27
• I don't know the solution but I always wanted to try Bruno Haible's libsigsegv and see if it can be made to work. But I have no idea if it can be done but I am more on the pessimistic side for this one. Good luck! – user21 Oct 14 '15 at 8:32
• ParallelEvaluate[subkernel] ? – rhermans Oct 14 '15 at 9:28
• @rhermans What I am looking for is how to prevent (or rather postpone) a kernel crash, not how to separate everything into a different process. – Szabolcs Oct 14 '15 at 10:20
• I was wondering if you got a chance to try out the libsigsegv? – user21 Nov 12 '15 at 9:58

If we all agree that bad things still might happen, then here is a start. I have set up a small repository with the code:

It currently works only on Linux (I have only tested it there), but I'm sure it is extendable to other systems as well.

The core idea is simply to create a handler for the signals that might be sent and return with an error from the library instead of loosing the kernel completely.

The function that produces a segfault is

static void do_segv() {
int *segv;
segv = 0;
*segv = 1;
}


and within Mathematica, you can call the library function with an integer where 0 means "do no segfault catching" and all other values set up a signal handler, call the do_segv and return with an error.

I didn't even bother to write the signal handler myself but stole it from SO. The whole function looks like this

DLLEXPORT int catchSegfault(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
mint i = MArgument_getInteger(Args[0]);
if (i == 0) {
do_segv();
return LIBRARY_NO_ERROR;
} else {
struct sigaction sa;

memset(&sa, 0, sizeof(sigaction));

sa.sa_flags = SA_NODEFER;
sa.sa_sigaction = handler;

sigaction(SIGSEGV, &sa, NULL);

if (setjmp(point) == 0) {
do_segv();
return LIBRARY_NO_ERROR;
} else {
return LIBRARY_MEMORY_ERROR;
}
}
}


It basically works because the signal handler

static void handler(int sig, siginfo_t *dont_care, void *dont_care_either) {
longjmp(point, 1);
}


jumps back to the last if condition returning 1 for setjmp which evaluates the else return. So when do_segv gets a SIGSEGV the handler is called and the library function immediately returns with an error.

I hope this gives at least a start.

Ahh, yes, before I forget it. Big thanks to Sasha for his awesome cmake plugin that lets you easily write library code in CLion.

• Thanks! But I'm running into problems. I tried this with CreateLibrary[{"library.c"}, "library", "TargetDirectory" -> NotebookDirectory[]] then f = LibraryFunctionLoad["library", "catchSegfault", {Integer}, "Void"] and function loads correctly. But both f[0] and f[1] cause kernel crash. Maybe I'm misusing it? – QuantumDot Jul 13 '17 at 3:17
• @QuantumDot On which OS are you working? – halirutan Jul 13 '17 at 3:38
• I should have said: I'm on Mac OS X 10.11. I noticed you stated that you tested it on Linux; I guess I'll have to do a little research to figure out how to make it work on Windows and Mac OS X as well. – QuantumDot Jul 13 '17 at 3:40
• @QuantumDot A painless method would be to write the same code in a short main that you can test on a console. If it works there but doesn't work inside Mathematica, we have a problem. – halirutan Jul 13 '17 at 3:43
• Ok good idea: I just did it on my console now (making catchSegfault a main) , and found that your code works as intended: i=0 produces Segmentation fault: 11 and i=1 returns without trouble. So I think there's a problem when running it from within Mathematica. – QuantumDot Jul 13 '17 at 3:48