5
$\begingroup$

I have a encountered a very weird behavior of Mathematica 11.3 when collecting and factoring large expressions. To be more precise, such operations often lead to an abrupt kernel crash, but the crashes are not deterministic: in most cases (something like 70%-80%) there will be a crash but sometimes the operation will finish successfully.

Since I was doing my calculations with FeynCalc, it took me quite some effort to isolate the code that can trigger the crash. The same code works without any issues on any other Mathematica version except for 11.3, so I'm fairly certain that the problem is present only in the latest version.

The code below is the minimal working example I managed to produce that would trigger the crash. It is still not truly "minimal", but this is because I do not understand myself, why the crashes actually occur. Together has something to do with it, but since you need to run the function collect several times with different parameters to produce a crash, I suspect that it is caused by some buffer overflow or a dangling pointer.

I could have just sent it to WRI, but there are several reasons why I decided to post it here first:

  1. It would be good to have some confirmations from other users. I'm running Mathematica 11.3 on Fedora Linux 28, so at the moment I do not know if the macOS and Windows versions are also affected.

  2. I have never encountered similar bugs before, so that I'm very curious to understand what is the reason for the behavior I am observing. Perhaps the kernel hackers here could diagnose the problem even without having access to the source code of Mathematica.

  3. As mentioned above, my MWE is not truly minimal and I'm afraid that the WRI employee who will receive my bug report would not be too enthusiastic to follow this issue. On the other hand, for me it is quite crucial that this bug gets fixed in the next release of Mathematica, since currently I simply cannot use version 11.3 in my research and have to stick with 11.0. My hope is that with the help of the people from this site we may narrow down this crash to just few lines of code that will make it easier for the WRI people to understand what goes wrong here.

OK, here is the example. Unfortunately, the input expression by itself is very large, so I used Compress to make it more tractable. Even in the compressed form it is still too large to be posted here, so I uploaded it to Pastebin.com

https://pastebin.com/9wuPPNcn

These are the functions that we need in order to trigger the crash. Originally, these were some FeynCalc functions, but I tried to remove everything that is not relevant for reproducing the crash. Hence, the functions look a bit weird, but they will not cause any abnormal behavior with any Mathematica version except for the latest one (11.3).

freeQ2[x_, {y_}] :=
    FreeQ[x, y];

freeQ2[x_, {y_, z__}] :=
  If[FreeQ[x, y],
   freeQ2[x, {z}],
   False
   ];

selectFree[a_, b__] :=
  Block[{dum1, dum2, select},
    select[x_, y_List] := Select[x, freeQ2[#, y] &];
    If[(Head[a] === Plus) || (Head[a] === Times),
            select[a, b],
            select[a dum1 dum2, b] /. {dum1 :> 1, dum2 :> 1}
        ]
   ];

selectNotFree[a_, b__] :=
  Block[{dum1, dum2, select},
    select[x_, y_List] := Select[x, ! freeQ2[#, y] &];
    If[(Head[a] === Plus) || (Head[a] === Times) || (Head[a] === List),
        select[a, b],
        select[a dum1 dum2, b] /. {dum1 :> 1, dum2 :> 1}
        ]
   ];

collect[expr_, vv_List] :=

  Block[{monomList, nx, tv, monomialHead, coeffArray, coeffHead},
    nx = expr;
    monomList = 
    Union[Select[
      vv, ((Head[#] =!= Plus) && (Head[#] =!= Times) && (! 
           NumberQ[#])) &]];

    nx = Expand[nx, Alternatives @@ monomList];
    nx = selectNotFree[nx, monomList];
    nx = Map[(selectFree[#, monomList] monomialHead[
         selectNotFree[#, monomList]]) &, nx];

    tv = Cases[nx, monomialHead[__], Infinity] // Union;
    coeffArray = CoefficientArrays[nx, tv];
    tv = (# /. monomialHead -> Identity) & /@ tv;
    nx = Sum[
     Dot[coeffHead[coeffArray[[i]]], 
      Sequence @@ Table[tv, {i - 1}]], {i, 2, Length[coeffArray]}];
    coeffHead[li_SparseArray] := Together /@ li;
    nx
   ];

Assuming that you saved the file from Pastebin to mmacrash.m.txt, you just need to evaluate the above code to define the function collect and then run the following in a Mathematica 11.3 notebook

SetDirectory[NotebookDirectory[]];
expr = Uncompress[Get["mmacrash.m.txt"]];

collect[expr, {int, c1, c2, d}];    
collect[expr, {int, c1, c2}];    
collect[expr, {int, c1, c2, d}];    
collect[expr, {int, c1, c2}];    
collect[expr, {int, c1, c2, d}];    
res = collect[expr, {int, c1, c2}];

If it does not crash after the evaluation has finished, just restart the kernel and hit Evaluate Notebook again: after the 3-4 evaluations there will be almost 100% probability to produce a crash.

EDIT: According to the WRI support [CASE:4160551] it is a bug that affects only the Linux version of Mathematica. They have forwarded the issue to the developers.

$\endgroup$
  • 1
    $\begingroup$ @DanielLichtblau Any chance that you could find time to look at this crash? Or should I better submit an official bug report? Seems like the SE community is not very much interested in looking into this, contrary to my original expectation :( I even suspect that this might be related to mathematica.stackexchange.com/questions/151650/… but of course for a non-developer it is difficult to tell. $\endgroup$ – vsht Sep 5 '18 at 6:29
  • 2
    $\begingroup$ Somehow I missed or forgot this. I do not offhand have a guess as to the issue. If it is in Factor or Together I'm sure it will find its way to me. $\endgroup$ – Daniel Lichtblau Sep 29 '18 at 12:11
  • $\begingroup$ Throw some print statements into your collect function. where does it crash? Between which 2 print statements? (I would do this myself but you seem to have a much easier time reproducing it than I do) $\endgroup$ – user6014 Sep 29 '18 at 17:23
  • $\begingroup$ Seems to be fixed in Mathematica 12.0. Many thanks to the WRI devs! $\endgroup$ – vsht Apr 24 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.