Does Abort not work for ParallelTable with SetSharedVariable?

Question

I find that often Abort (command-. on Mac) sometimes does not work if ParallelTable is running. Here's a simple example:

SetSharedVariable[foo]
ParallelTable[
 foo = j^j^j;
 , {j, 1, 200}]

This starts spitting Overflow errors and can't be aborted. I find however that if I don't use SetSharedVariable, then Aborting is possible. What's going on?

(I use Mathematica 11.0.0.0 on a MacBook)

Answer 1

Unlike in my comment and my first version of this answer I now realized that your computation of j^j^j for integers is actually becoming very long lasting for j>8, so my previous assumptions do not hold. I still think that part of my answer is valid and can actually even be tested with the following code:

ParallelTable[
  Module[{r},
    Print["Starting: ", j];
    r = j^j^j;
    Print["Finished: ", j];
    Print["Sending: ", j];
    shared = r;
    Print["Done: ", j, ", ", ByteCount[r]];
  ], {j, 1, 10}];

You will find that it looks like after a short initial period where the small numbers are computed (and during which my original assumption might apply) there is premanently one Kernel sending a (large) result back to the master. So my assumption of permanently held locks for the shared variable might be still be correct, although for different reasons. Note that the following code is easily abortable despite the use of a shared variable, as we now only send back small amounts of data which works fast:

ParallelTable[
  Module[{r},
    Print["Starting: ", j];
    r = j^j^j;
    Print["Finished: ", j];
    Print["Sending: ", j];
    shared = ByteCount[r];
    Print["Done: ", j, ", ", ByteCount[r]];
  ], {j, 1, 10}];

Old answer with incorrect assumptions

Your code will make all kernels try to write to the shared variable at a very high pace (as they do not have a whole lot work to do). The writing of a shared variable will always mean some kind of locking needs to be implemented to avoid inconsistencies. The net result will be that during the execution of that loop I would expect the kernels to mainly be busy waiting to get ahold of the variables lock. And of course the shared variable will permanently be locked. While a variable is locked it seems reasonable that at some place aborts will be prevented during a lock is hold to avoid it not being freed.

I have no definite workaround but of course would suggest to avoid extensive usage of SetSharedVariable, which would also be a good idea if you want to see good speedup. You could also look into this question, where a case is discussed where sending back a large result took an unreasonable amount of time and I think that something similar could also be the case in that particular example or your real code.

As whether every situation where an immediate abort does not work will classify as a bug I'm not so sure. I think we might agree that it is OK to need three aborts to stop the following code:

Do[CheckAbort[Pause[10^9], $Aborted], {3}]

But is it a bug in Mathematica or the user code that the following (better don't execute) can not be aborted?

While[True,CheckAbort[Pause[10^9], $Aborted]]

Of course in the case of the ParallelTable code the abort behavior is much less explicit and not even well documented. I also think that it should be possible to abort the execution on user demand despite the lower level protection of shared variable locks in one way or another. The current behavior certainly is inconvenient and user-unfriendly and also seems to be version dependent and even affects examples in WRI's own documentation: the first example in the Application section of the SetSharedVariable documentation works for me as intended with 9.0.1, it hangs with 10.4.1 while it can be aborted with some error messages in 11.0.1. So it looks like the behavior you see might indeed not be what is intended and it is worth to give WRI some "product feedback"...