Parallelize Map and ParallelMap - Mathematica Stack Exchange most recent 30 from mathematica.stackexchange.com 2019-11-11T23:18:36Z https://mathematica.stackexchange.com/feeds/question/138476 https://creativecommons.org/licenses/by-sa/4.0/rdf https://mathematica.stackexchange.com/q/138476 37 Parallelize Map and ParallelMap Mher https://mathematica.stackexchange.com/users/8318 2017-02-23T16:05:41Z 2019-04-15T14:13:35Z <p><strong>Bug introduced in 8.0 and fixed in 11.2</strong></p> <hr> <p>It is stated in the documentation that</p> <blockquote> <p><code>Parallelize[Map[f, expr]]</code> is equivalent to <code>ParallelMap[f, expr]</code>.</p> </blockquote> <p>But what about these examples?</p> <pre><code>ParallelMap[#^2 &amp;, f[x] + g[y]] (* f[x]^2 + g[y]^2 *) Parallelize[Map[#^2 &amp;, f[x] + g[y]]] (* f[x^2] + g[y^2] *) </code></pre> https://mathematica.stackexchange.com/questions/138476/-/138490#138490 31 Answer by Szabolcs for Parallelize Map and ParallelMap Szabolcs https://mathematica.stackexchange.com/users/12 2017-02-23T19:16:43Z 2017-02-24T13:10:12Z <p>I did a bit of debugging to find the cause of the problem. After I found it, the problem no longer seems as outrageous as it looked at first sight.</p> <p>The root cause of the problem is the following property of <code>Plus</code>:</p> <pre><code>Plus[x] (* x *) </code></pre> <p>This means that unlike <code>List</code>, <code>Plus</code> cannot be used as a container that can be split into smaller parts and then put together again.</p> <p>This works:</p> <pre><code>Join[Plus[a, b], Plus[c, d]] (* a + b + c + d *) </code></pre> <p>This does not:</p> <pre><code>Join[Plus[a], Plus[c, d]] (* Join[a, c + d] *) </code></pre> <p>Partitioning is the first step to parallelization—each partition (or batch) will be sent to a different subkernel. You have precisely two elements to evaluate (<code>f[x]</code> and <code>g[x]</code>), so they get partitioned into two batches of length 1 each. <code>Plus[f[x], g[x]]</code> ends up split into <code>Plus[f[x]]</code> and <code>Plus[g[x]]</code>. At one point these are (incorrectly) allowed to evaluate to <code>f[x]</code> and <code>g[x]</code>.</p> <h3>More detailed analysis</h3> <p>The literal expression the system ends up constructing (and submitting for parallel evaluation) is:</p> <pre><code>HoldComplete[ (sq /@ # &amp;)[Unevaluated[Plus[f[x]]]], (sq /@ # &amp;)[Unevaluated[Plus[g[x]]]] ] </code></pre> <p><code>sq</code> here is <code>#^2&amp;</code>—I am going to use <code>sq</code> from now on to make it easier to follow what is happening. The two elements within <code>HoldComplete</code> are the two size-1 batches, with processing ready to be applied to them.</p> <p>Now watch carefully what happens if we evaluate one of these elements:</p> <ul> <li><p>First, the <code>Unevaluated</code> gets stripped.</p></li> <li><p>Then we get <code>sq /@ Plus[f[x]]</code></p></li> <li><p>Now the <code>Plus</code> evaluates because it has a single argument. If it had at least two, it wouldn't. We get <code>sq /@ f[x]</code></p></li> <li><p>And then we get <code>f[sq[x]]</code> and finally <code>f[x^2]</code>.</p></li> </ul> <p>An interesting note is that if we had <code>Map[sq]</code> instead of <code>sq /@ # &amp;</code>, then no further evaluation would take place after the <code>Unevaluated</code> gets stripped, and the problem would be averted (<em>hint: perhaps this could be a good fix</em>).</p> <p>So if you thought that <code>Map[f]</code> and <code>f /@ #&amp;</code> were the same thing, here's one example that proves them different.</p> <h3>Why is there a difference between <code>ParallelMap[...]</code> and <code>Parallelize[Map[...]]</code>?</h3> <p><code>ParallelMap[f, arg]</code> effectively translates to</p> <pre><code>ParallelCombine[ Function[e, Map[f,Unevaluated[e]]], arg ] </code></pre> <p><sup>see <code>Combine.m</code>, line 324</sup></p> <p><code>Parallelize[Map[f, arg]]</code> effectively translates to</p> <pre><code> ParallelCombine[ Map[f, #]&amp;, arg ] </code></pre> <p>The latter lacks an <code>Unevaluated</code>, which is the root of the problem.</p> <p><sup>see <code>Evaluate.m</code>, line 137</sup></p>