How can multiple kernels update a shared Association object?

Question

In the examples below, the variable status is an Association object that is supposed to be shared by three concurrent kernels.

The assignment status[$KernelID] = 1 fails if I run it within ParallelEvaluate:

status = <||>;
SetSharedVariable[status];
ParallelEvaluate[status[$KernelID] = 1;];
status

Set::noval: Symbol status in part assignment does not have an immediate value.

Set::noval: Symbol status in part assignment does not have an immediate value.

Set::noval: Symbol status in part assignment does not have an immediate value.

<||>

An equivalent assignment works fine if I run it outside of ParallelEvaluate, as shown below. (NB: In the snippet below, I use ParallelEvaluate only to get the values of $KernelID for the various kernels; the assignments to status happen outside ParallelEvaluate).

status = <||>;
SetSharedVariable[status];
Scan[(status[#] = 1) &, ParallelEvaluate[$KernelID]];
status

<|1 -> 1, 2 -> 1, 3 -> 1|>

Is there a way to share an Association object among all kernels in such a way that each kernel can modify the object?

FWIW, I am immediately interested in the use-case where each kernel assigns only to the key corresponding to its $KernelID.

---

UPDATE: FWIW, the problem seems to be specific to Associations; a similar scenario using a list instead of an association works fine:

statusList = ConstantArray[Null, Max[ParallelEvaluate[$KernelID]]];
SetSharedVariable[statusList];
ParallelEvaluate[statusList[[$KernelID]] = 1;];
statusList

{1, 1, 1}

(This looks more and more like a bug to me... In this case, I suppose I should specify that I'm using 10.4.1.0 Linux x86 (64-bit).)

Answer 1

So I'm pretty sure this is a bug but here's why I think it happens and here's a workaround:

We'll do the setup:

status = <||>;
SetSharedVariable[status];

Then check the OwnValues:

In[354]:= ParallelEvaluate[OwnValues[status]]

Out[354]= {{HoldPattern[status] :> 
   Parallel`Client`CallBack[status]}, {HoldPattern[status] :> 
   Parallel`Client`CallBack[status]}, {HoldPattern[status] :> 
   Parallel`Client`CallBack[status]}, {HoldPattern[status] :> 
   Parallel`Client`CallBack[status]}}

From past digging in the Parallel`Developer context I think that's just a handler that the subkernel sees and then realizes "oh, I should pull the value from the parent kernel". This then breaks the sym[key]=val assignment pattern of associations. Of course there are other handlers in place to do setting synchronization, so I thought it might be worth trying AssociateTo. And that does in fact work:

In[355]:= ParallelEvaluate[AssociateTo[status, $KernelID -> 1]]

Out[355]= {Null, Null, Null, Null}

In[356]:= status

Out[356]= <|1 -> 1, 2 -> 1, 3 -> 1, 4 -> 1|>

This also seems to makes sense for why Part can do the assignment. It's a) older and b) a proper symbol that the subkernel can check for. But of course I know nearly nothing about parallel kernels so that's just how I conceive of it.

In any case, to summarize, I think it's a bug and AssociateTo is a work-around.