3
$\begingroup$

I see a weired behavior in my MM 10.3 version. Can someone reproduce it or explain what is going on? The code is easy:

a = RandomReal[{1, 100}, 10000000];
d = ParallelTable[a[[i]], {i, 10}];
UnsetShared[a]

{a}

After that a is still known to the master (of course). Now I want to do the ParallelTable again.

d = ParallelTable[a[[i]], {i, 10}];

Part::partd: Part specification a[[1]] is longer than depth of object.

That is strange as in the first call of ParallelTable MM distributes automatically. Anyhow I could do it by hand using: DistributeDefinitions[a] but the answer is {}. So the a is not distributed. As a consequence the result is still the same:

d = ParallelTable[a[[i]], {i, 10}];

Part::partd: Part specification a[[1]] is longer than depth of object.

Only when I redo a = RandomReal[{1, 100}, 10000000]; (which should not be necessary as a is still known the master kernel) I can distribute a again. Is there anything apart from Distribute and UnsetShared that I can do to be able to distribute a again?

$\endgroup$
  • $\begingroup$ You get the error message, but doesn't d still return the correct answer? See the result here $\endgroup$ – Jason B. Feb 4 '16 at 10:15
  • $\begingroup$ @JasonB The explanation is that using UnsetShared on something that has not been shared at the beginning messes up the internal bookeeping used for sharing and distributing definitions. The solution is not to use UnsetShared on something that has not been previously been shared. $\endgroup$ – Szabolcs Feb 4 '16 at 10:27
  • $\begingroup$ I am not going to tag this as a bug, but I did report it to Wolfram. I am going to leave the decision about tagging to others (or will do it if Wolfram confirms that they consider it a bug). $\endgroup$ – Szabolcs Feb 4 '16 at 10:51
  • $\begingroup$ @Jason B: Indeed I get the same result even though the error is issued. That is even more weired. $\endgroup$ – Eisbär Feb 4 '16 at 10:58
5
$\begingroup$

This happens because you never set a as a shared variable before using UnsetShared. That is, you were using UnsetShared in an incorrect manner.

In short: make sure you never use UnsetShared on a symbol unless it was shared first!

Mathematica in general is pretty forgiving (unlike languages like C) and won't make it possible to mess up its internal state for good (unless you explicitly poke around in its insides). This is one of those cases when it is being less forgiving, and a single incorrect use of UnsetShared can induce a persistent bad state.

Observe what happens during correct usage

We set a on the main kernel.

In[1]:= a = 1;

Using ParallelDo automatically distributes a.

In[2]:= ParallelDo[Print[a], {4}]

(kernel 4) 1

(kernel 3) 1

(kernel 2) 1

(kernel 1) 1

a is now recorded in the list of distributed symbols. This list is used to prevent distributing the same symbol every single time it's used even if its value hasn't changed. It is an optimization measure.

In[3]:= Parallel`Developer`$DistributedDefinitions    
Out[3]= {Hold[a]}

Setting a as shared removes it from the list of distributed symbols to prevent bad interactions between the sharing and distribution mechanisms. It also prevents further distribution through other means.

In[4]:= SetSharedVariable[a]

In[5]:= Parallel`Developer`$DistributedDefinitions    
Out[5]= {}

After "unsharing" a, it gets removed from the shared list ($SharedVariables), and it will be available for distribution again.

In[6]:= UnsetShared[a]    
Out[6]= {a}

ParallelDo now does what we expect: it distributes a if necessary and evaluates as usual.

In[7]:= ParallelDo[Print[a], {4}]

(kernel 4) 1

(kernel 3) 1

(kernel 2) 1

(kernel 1) 1

Observe what happens during incorrect usage

In[1]:= a = 1;

In[2]:= ParallelDo[Print[a], {4}]

(kernel 4) 1

(kernel 3) 1

(kernel 2) 1

(kernel 1) 1

In[3]:= Parallel`Developer`$DistributedDefinitions
Out[3]= {Hold[a]}

So far so good.

But now we try to "unshare" a without it having been shared first. This means that the distribution mechanism has never been disabled for a. But now UnsetShared clears its values in the subkernels.

In[4]:= UnsetShared[a]
Out[4]= {a}

On the main kernel it appears that a is already distributed. It's still listed in $DistributedDefinitions. Unless its value changes on the main kernel, it won't get redistributed. The main kernel does not know that the subkernel values of a are inconsistent with the main kernel values.

In[5]:= Parallel`Developer`$DistributedDefinitions
Out[5]= {Hold[a]}

Even if we tried to distribute again (which ParallelDo does automatically), a remains undefined on the subkernels. That's because the main kernel thinks that its value is already consistent between all kernels.

In[6]:= ParallelDo[Print[a], {4}]

(kernel 4) a

(kernel 3) a

(kernel 2) a

(kernel 1) a

If you accidentally get yourself into this situation, you can reset the distribution mechanism like this:

In[7]:= Parallel`Developer`ClearDistributedDefinitions[]

In[8]:= ParallelDo[Print[a], {4}]

(kernel 4) 1

(kernel 3) 1

(kernel 2) 1

(kernel 1) 1

Note that Parallel`Developer`$DistributedDefinitions is not a user-settable list (it's not even a list under the hood, it just looks like it), so do not try to remove Hold[a] from there.

Finally

I am not sure if I should call this a bug because the problem is only triggered if the user makes a mistake. But it would be much better if UnsetShared were a little smarter and would refuse to "unshare" things that haven't already been shared. I don't see why it couldn't do this given that $SharedVariables and $SharedFunctions exist. I am going to send a suggestion to Wolfram Support about this.

UnsetShared already behaves nicely if it is given a string instead of a symbol. UnsetShared["a"] will only "unshare" a if it has already been shared. UnsetShared[a] will misbehave, as you observed.

$\endgroup$
  • $\begingroup$ So we have the pair of SetSharedVariable and UnsetShared. The pendant for DistributeDefinition is ParallelDeveloperClearDistributedDefinitions[] (UnsetDefinition would be nice :)). So the thing going wrong here is actually that UnsetShared clears "a" from some "list" (it is neither $SharedVariables nor ParallelDeveloper$DistributedDefinitions) that makes ParallelTable issue the error even though "a" was not unset from the kernels. $\endgroup$ – Eisbär Feb 4 '16 at 11:21
  • $\begingroup$ @Eisbär The problem is simply that the two functions were not used in a pair. The mechanism by which the problem manifests itself is not that UnsetShared clears a from a bookkeeping list. It is that you didn't use SetSharedVariable so a never got cleared from the distributed list! But the solution is not to try to edit the internal state, as that can very easily break things. The solution is to always use SetSharedVariable and UnsetShared in pairs. $\endgroup$ – Szabolcs Feb 4 '16 at 11:24
  • $\begingroup$ Something that I noticed when looking at RAM consumption. UnsetShared and ParallelDeveloperClearDistributedDefinitions[] free the same amount of memory. However, after a ParallelDeveloperClearDistributedDefinitions[] the memory consumption jumps up again while after UnsetShared it doesn't. In both cases the list ParallelTable makes (d) is the same. Only last mentioned procedure issues an error. How can UnsetShared free the memory and the ParallelTable after that giving still the same result? $\endgroup$ – Eisbär Feb 4 '16 at 11:28
  • $\begingroup$ @Szablocs: The missing bit for me was ParallelDeveloperClearDistributedDefinitions[] as ParallelTable internally uses DistributeDefinitions for "a" and not SetSharedVariable. So UnsetShared was the only thing coming close to what I needed to free the memory after ParallelTable. That was wrong. $\endgroup$ – Eisbär Feb 4 '16 at 11:32
  • 1
    $\begingroup$ @Eisbär I don't fully understand your question, but I think it is related to this phenomenon: let's work with ParallelEvaluate which (unlike ParallelTable) does not automatically distribute definitions. In a newly started kernel set a=1; and try ParallelEvaluate[a+a]. It appears to work, but the calculation wasn't done on the subkernels. Instead each subkernel returned a symbolic a+a to the main kernel, where that expression evaluated further. Compare ParallelEvaluate[Print[a+a]] to ParallelEvaluate[a+a]. Because of this some calculations appear to work but are in fact... $\endgroup$ – Szabolcs Feb 4 '16 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.