Highest voted questions tagged parallelization - Mathematica Stack Exchange most recent 30 from mathematica.stackexchange.com 2019-10-21T06:01:21Z https://mathematica.stackexchange.com/feeds/tag?tagnames=parallelization&sort=votes https://creativecommons.org/licenses/by-sa/4.0/rdf https://mathematica.stackexchange.com/q/104 61 Speeding up this fractal-generating code Isaac https://mathematica.stackexchange.com/users/7 2012-01-18T04:39:46Z 2019-04-07T02:29:49Z <p>I used the code below (which is a sample from <a href="https://gist.github.com/497210" rel="noreferrer">this gist containing more similar code</a>) in <a href="https://math.stackexchange.com/a/1107/72">my answer to my own question about Mandelbrot-like sets for functions other than the simple quadratic on Math.SE</a> to generate this image:</p> <p><img src="https://i.stack.imgur.com/pL8nx.png" alt="graphic"></p> <pre><code>cosineEscapeTime = Compile[{{c, _Complex}}, Block[{z = c, n = 2, escapeRadius = 10 \[Pi], maxIterations = 100}, While[And[Abs[z] &lt;= escapeRadius, n &lt; maxIterations], z = Cos[z] + c; n++]; n]] Block[{center = {0.5527, 0.9435}, radius = 0.1}, DensityPlot[ cosineEscapeTime[x + I y], {x, center[] - radius, center[] + radius}, {y, center[] - radius, center[] + radius}, PlotPoints -&gt; 250, AspectRatio -&gt; 1, ColorFunction -&gt; "TemperatureMap"]] </code></pre> <p>What could I do to improve the speed/time-efficiency of this code? Is there any reasonable way to parallelize it? (I'm running Mathematica 8 on an 8-core machine.)</p> <hr> <p><strong><em>edit</em></strong> Thanks all for the help so far. I wanted to post an update with what I'm seeing based on the answers so far and see if I get any further refinements before I accept an answer. Without going to hand-written C code and/or OpenCL/CUDA stuff, the best so far seems to be to use <code>cosineEscapeTime</code> as defined above, but replace the <code>Block[...DensityPlot[]]</code> with:</p> <pre><code>Block[{center = {0.5527, 0.9435}, radius = 0.1, n = 500}, Graphics[ Raster[Rescale@ ParallelTable[ cosineEscapeTime[x + I y], {y, center[] - radius, center[] + radius, 2 radius/(n - 1)}, {x, center[] - radius, center[] + radius, 2 radius/(n - 1)}], ColorFunction -&gt; "TemperatureMap"], ImageSize -&gt; n] ] </code></pre> <p>Probably in large part because it parallelizes over my 8 cores, this runs in a little under 1 second versus about 27 seconds for my original code (based on <code>AbsoluteTiming[]</code>).</p> https://mathematica.stackexchange.com/q/2886 45 Transferring a large amount of data in parallel calculations Szabolcs https://mathematica.stackexchange.com/users/12 2012-03-13T17:42:48Z 2017-04-20T15:17:22Z <p><strong>Bug fixed in version 11.1</strong> <br> Functions like <code>MemberQ</code>, <code>FreeQ</code>, etc. no longer unpack. Yay!</p> <hr> <p>This question is inspired by <a href="https://mathematica.stackexchange.com/a/2849/12">one of @whuber's answers</a></p> <p>Consider the following code:</p> <pre><code>μ = RandomReal[{0, 1}, 100]; Σ = DiagonalMatrix[Exp[RandomReal[{0, 1}, 100]]]; AbsoluteTiming[ RandomVariate[MultinormalDistribution[μ, Σ], 400000];] </code></pre> <p>It runs in 3.5 seconds here.</p> <p>Now let's parallelize it:</p> <pre><code>LaunchKernels[] AbsoluteTiming[ Join @@ ParallelTable[ RandomVariate[MultinormalDistribution[μ, Σ], 200000], {2}];] </code></pre> <p>This runs in 6.3 seconds on a 2-core machine---much <em>slower</em>. It also uses a lot of memory (which I checked using a process monitor).</p> <p>Now let's suppress returning the results from the subkernels by including a semicolon:</p> <pre><code>AbsoluteTiming[ Join @@ ParallelTable[ RandomVariate[MultinormalDistribution[μ, Σ], 200000];, {2}];] </code></pre> <p>This one runs in 2.6 seconds---a speedup.</p> <p>What is happening here? Why is the calculation that returns the result so much slower? Is it a general rule with parallel calculations that returning even moderately large data tends to lead to a significant slowdown? Is the slowdown due to MathLink's performance? Is there anything one could do to avoid the slowdown? </p> <p><strong>Warning:</strong> This might eat all your memory and force your system to swap! This computer has 6 GB and everything was fine. If you have less memory, reduce the amount of data a bit.</p> <hr> <h2>Solution</h2> <p>@Oleksandr's excellent analysis showed that the performance bottleneck is <code>MemberQ</code>, in particular that it unpacks all arrays inside the expression tested. This is completely unnecessary, and it's possible to define a more efficient (though more limited) version of <code>MemberQ</code>:</p> <pre><code>memberQ[list_, form_] := Or @@ (MatchQ[#, form] &amp; /@ list) </code></pre> <p>Note that <code>MemberQ</code> only tests at level 1 by default (unlike <code>FreeQ</code> which tests at all levels). This made it easy to re-implement the two-argument form of <code>MemberQ</code>.</p> <p>We can temporarily change <code>MemberQ</code> while executing parallel operations:</p> <pre><code>ClearAll[fix] SetAttributes[fix, HoldAll] fix[expr_] := Block[{MemberQ = memberQ}, expr] fix@AbsoluteTiming[ Join @@ ParallelTable[ RandomVariate[MultinormalDistribution[μ, Σ], 200000], {2}];] </code></pre> <p>This runs in 3.0 seconds now, a huge improvement.</p> <p>This is just an illustration of how to fix the performance problem, but the code I showed here is not completely safe to use in its current form.</p> <p>Some notes:</p> <ul> <li><p>Changing builtins is always risky, and <a href="https://mathematica.stackexchange.com/questions/1495/how-can-i-speed-up-image-importing">can easily cause problems</a> </p></li> <li><p>I used <code>Block</code> to localize the change, which reduces the risk. Note that <code>Block</code> will not affect calculations in the parallel kernels, so if <code>fix</code> is used <em>only</em> on the parallelization functions, in the form <code>fix@ParallelTable[...]</code>, then it will only have an effect for these functions, but not for the code that is being parallelized. This reduces the risk further.</p></li> <li><p>I did not implement the 3-argument form of <code>MemberQ</code>. If this is used anywhere in the parallel tools, <code>fix</code> will break things. It'd take a bit more work to correctly implement this too, preferably just falling back to the builtin <code>MemberQ</code> for this case. There may always be some undocumented behaviour of <code>MemberQ</code> which we are not aware of and which differs from <code>memberQ</code>.</p></li> <li><p>I did not implement short circuiting, so <code>memberQ</code> will be slower in some cases. This can be fixed as well.</p></li> </ul> <p>These potential problems can largely be fixed with a bit of work, and I believe this method can work well for fixing this particular performance problem of parallel calculations.</p> https://mathematica.stackexchange.com/q/1008 43 Is it possible to speed up ContourPlot on multi-core machines? faleichik https://mathematica.stackexchange.com/users/219 2012-01-30T20:47:49Z 2016-03-22T14:52:06Z <p>It is not very difficult to face a function for which <code>ContourPlot</code> works too slow. And it seems natural that this function can be parallelized well. Anyway, naive <code>Parallelize@ContourPlot</code> produces "<em>ContourPlot[...] cannot be parallelized;</em> "...</p> <p>So, is it possible to parallelize ContourPlot?</p> https://mathematica.stackexchange.com/q/104842 43 What would you ask Mathematica to do on a big system? pvanbijnen https://mathematica.stackexchange.com/users/21447 2016-01-25T19:06:47Z 2016-02-04T15:48:43Z <p>Our hardware supplier has given us the opportunity to test Mathematica on a big Windows system. The specs will be 1 to 4 TB of working memory and 12 sockets with 18 cores each. I got 1 full week of computer time to do my testing. The question is what should I ask Mathematica to do because it is more a test of Mathematica and not a test of the hardware. </p> <p>I could load a complete database in memory using the association structure and then test search and consolidate questions.</p> <p>Another option is to run an algorithm that benefits from parallel computations. </p> <p>The goal is to write a test plan on what to test and how to test it.</p> <p>I welcome your suggestions on what to evaluate in Mathematica to measure how useful many processors and almost unlimited working memory is?</p> <p>Kind regards,</p> <p>Pieter van Bijnen</p> <p>note: it is one system (no grid involved)</p> https://mathematica.stackexchange.com/q/3674 43 Are built-in Mathematica functions already parallelized? Guillochon https://mathematica.stackexchange.com/users/805 2012-03-29T17:18:48Z 2013-04-03T22:27:14Z <p>I've been noticing something strange since updating to Mathematica 8, and that is that occaisionally I'll see that the MathKernel is using up to 800% CPU in my Activity Monitor on OS X (I have 8 cores). I have no Parallel calls whatsoever, and this is in a single kernel, not across multiple kernels. My code is pretty much only Interpolates, Maps, Do loops, and plotting routines.</p> <p>I'm curious if some of the built-in Mathematica routines are in fact already parallel, and if so, which ones?</p> https://mathematica.stackexchange.com/q/1259 43 Parallelize evaluation of function with memoization Volker https://mathematica.stackexchange.com/users/386 2012-02-03T17:51:11Z 2016-09-01T14:47:33Z <p>I have a complicated function that I need multiple times, so I want to memoize it and have the first evaluation done in parallel. Unlike in my example below it's not a continuous function, so interpolation is not an option. (In fact, it's values are also functions.) The naive approach clearly does not work, because the memoized value is then only known on the kernel it was evaluated on:</p> <pre><code>LaunchKernels; f[x_] := f[x] = (Pause; N[Sin[x]]); (*Expensive calculation*) ParallelDo[f[x], {x, 3}]; ParallelEvaluate[AbsoluteTiming[f]] (* ==&gt; {{3.000632, 0.841471}, {0.000024, 0.841471}} *) </code></pre> <p>I believe I found a workaround by doing something like this:</p> <pre><code>f[x_] := (Pause; N[Sin[x]]); (*Expensive calculation - NO memoization*) t = ParallelTable[f[x], {x, 3}]; Do[f[x] = t[[x]], {x,3}]; </code></pre> <p>Using <code>SetSharedFunction[f]</code> before <code>ParallelDo</code> also yields a non-optimal result:</p> <pre><code>{{0.012051, 0.841471}, {0.012202, 0.841471}} </code></pre> <p>0.01 s is still a long time to look up a value (see above, it should be &lt; 1 ms). Is there something more elegant or do I have to keep it like this?</p> <p>Edit: Just to be clear, the workaround above without Shared Functions works, it runs in parallel and the main kernel knows the values afterwards, but it strikes me as an ugly hack. I was wondering if there was an "official" solution.</p> https://mathematica.stackexchange.com/q/48295 41 Why won't Parallelize speed up my code? Szabolcs https://mathematica.stackexchange.com/users/12 2014-05-21T21:12:23Z 2017-04-08T19:50:46Z <p>What reasons are there that can cause <a href="http://reference.wolfram.com/language/guide/ParallelComputing.html">parallelized</a> Mathematica code not to run with full performance?</p> https://mathematica.stackexchange.com/q/54553 41 Mma 10: Half the parallel power (Macs)? wolfies https://mathematica.stackexchange.com/users/898 2014-07-11T13:42:22Z 2019-10-16T13:02:29Z <p>Here is a comparison of the parallel kernels launched under <em>Mathematica</em> under v9 and v10, on the same identical current 2014 R2-D2 Mac Pro ... </p> <p>[ Update: Valerio has commented that the same issue arises on the Macbook Air.] </p> <h1>Under v9.01</h1> <pre><code>$ProcessorCount </code></pre> <blockquote> <p>12</p> </blockquote> <p>Issuing:</p> <pre><code> LaunchKernels[] </code></pre> <p>... launches 12 kernels, and actually uses them ... notice that the <code>ParallelTable</code> is 12 times the speed of <code>Table[]</code> for this construct:</p> <pre><code>In:= Table[Pause; f[i], {i, 12}] // AbsoluteTiming </code></pre> <blockquote> <p>Out= {12.003106, {f<a href="https://i.stack.imgur.com/lfrC8.png" rel="noreferrer">1</a>, f<a href="https://i.stack.imgur.com/2rz8f.png" rel="noreferrer">2</a>, f, f, f, f, f, f, f, f, f, f}}</p> </blockquote> <pre><code>In:= ParallelTable[Pause; f[i], {i, 12}] // AbsoluteTiming </code></pre> <blockquote> <p>Out= {1.010648, {f<a href="https://i.stack.imgur.com/lfrC8.png" rel="noreferrer">1</a>, f<a href="https://i.stack.imgur.com/2rz8f.png" rel="noreferrer">2</a>, f, f, f, f, f, f, f, f, f, f}} </p> </blockquote> <p>So, to perform the same operation, the parallel result under v9 is 12 times the speed of the single kernel result.</p> <h1>Under v10 -- half my potential processing power has gone</h1> <pre><code>$ProcessorCount </code></pre> <blockquote> <p>6</p> </blockquote> <p>... down from 12 - even though I am running on the identical machine. Now, I know that my Mac Pro actually has 6 processors, and each runs 2 threads ... and under v9, that yielded 12 processor kernels for Mma 9 ... but under v10, it is only yielding 6 kernels ... ON THE SAME MACHINE. And this has real effects ... it effectively reduces by 50% the maximum potential power of my Mac: </p> <pre><code> LaunchKernels[] </code></pre> <p>... launches 6 kernels (not 12 kernels as under v9).</p> <p>Compare the performance:</p> <pre><code> In:= ParallelTable[Pause; f[i], {i, 12}] // AbsoluteTiming </code></pre> <blockquote> <p>Out= {2.009933, {f<a href="https://i.stack.imgur.com/lfrC8.png" rel="noreferrer">1</a>, f<a href="https://i.stack.imgur.com/2rz8f.png" rel="noreferrer">2</a>, f, f, f, f, f, f, f, f, f, f}}</p> </blockquote> <p>So, under the new v10, I am getting half the parallel performance here and half the kernels that I got under v9. Even more perplexing is that this worked fine in an earlier pre-release version of v10.</p> <p>I am very confused. Anyone have any ideas how I can get my missing kernels back? Or why a decision may have been made to hobble the performance of the Mac Pro under v10?</p> <h1>Addendum</h1> <p>Just noticed that if I go to:</p> <ul> <li><code>Evaluation Menu -&gt; Parallel Kernel Configuration</code></li> </ul> <p>... the automatic setting for:</p> <ul> <li><code>Number of kernels to use: is set to: Automatic</code> (which Mma sets to 6)</li> </ul> <p>If I change this to: </p> <ul> <li><code>Manual setting</code></li> </ul> <p>and set it to 12 ... then it seems to use 12.</p> <p>But I am still confused as to why, if <em>Mathematica</em> 10 can actually support 12 kernels on the machine, ... why would Wolfram set it to use only half of them by default, when v9 supported all of them by default?</p> <h1>Reply to Szabolcs: real-world test</h1> <p>Szabolcs suggests below that <em>Mathematica</em> may not practically use more kernels than physical cores, even if your processor supports virtual cores ... so there is no real difference. In reply, here is a quick timing test of a real-world application (kernel density estimation) from the <em>mathStatica</em> benchmarking test suite. The task is to plot 12 kernel density estimates, corresponding to 12 different bandwidths.</p> <pre><code>bandwidths = {.2, .35, .45, .55, .65, 1, 1.5, 2, 2.2, 2.5, 3, 3.2}; </code></pre> <p><a href="https://i.stack.imgur.com/2rz8f.png" rel="noreferrer"><img src="https://i.stack.imgur.com/2rz8f.png" alt="enter image description here"></a></p> <p>Here are the results running under:</p> <ul> <li>v9 (default: 12 kernels): 3.38 seconds</li> <li>v10 (default: 6 kernels): 9.53 seconds</li> <li>v10 (manual overide to 12 kernels): 7.46 seconds</li> </ul> <p>I don't know what has changed to cause such a performance hit under v10 ... but even so, that is not the point. The point is that the v10 default kernel setting fails to take advantage of the power of the Mac Pro ... and results in worse performance in a typical parallel-processing application. </p> <h1>More extensive real-world test:</h1> <p><em>Update: 1 August 2014</em></p> <p>I have now had the opportunity to run the full <em>mathStatica</em> (primarily symbolic) benchmark suite under both:</p> <ul> <li>the default v10 parallel setting (6 kernels)</li> <li>the manual override v10 setting (12 kernels)</li> </ul> <p>Here are the results:</p> <p><a href="https://i.stack.imgur.com/lfrC8.png" rel="noreferrer"><img src="https://i.stack.imgur.com/lfrC8.png" alt="enter image description here"></a></p> <p>The results fall into 2 categories:</p> <ul> <li><p><em>For problems that have more than 6 separate components to them</em>: ... For such problems, using 12 kernels is ALWAYS unambiguously faster, and significantly so. </p></li> <li><p><em>For problems that have 6 or less separate components</em>: ...For instance, Examples 7 and 9 can only be broken down into 2 symbolic components, so the benefits of parallelism max out with 2 kernels. In these cases, the 6 automatic kernels case is sometimes marginally faster than the 12 kernel case (presumably due to running overheads etc) ... but the difference is tiny, and essentially unnoticeable.</p></li> </ul> <p><strong>In summary:</strong> for problems that CAN benefit from more than 6 kernels, the default Mma 10 (automatic) setting of 6 kernels on a Mac Pro appears to be sub-optimal, and fails to take advantage of the full capability of the machine. This problem is new to v10, and does not occur under v9.</p> https://mathematica.stackexchange.com/q/138476 37 Parallelize Map and ParallelMap Mher https://mathematica.stackexchange.com/users/8318 2017-02-23T16:05:41Z 2019-04-15T14:13:35Z <p><strong>Bug introduced in 8.0 and fixed in 11.2</strong></p> <hr> <p>It is stated in the documentation that</p> <blockquote> <p><code>Parallelize[Map[f, expr]]</code> is equivalent to <code>ParallelMap[f, expr]</code>.</p> </blockquote> <p>But what about these examples?</p> <pre><code>ParallelMap[#^2 &amp;, f[x] + g[y]] (* f[x]^2 + g[y]^2 *) Parallelize[Map[#^2 &amp;, f[x] + g[y]]] (* f[x^2] + g[y^2] *) </code></pre> https://mathematica.stackexchange.com/q/1548 36 Monitor doesn't work with ParallelTable Valerio https://mathematica.stackexchange.com/users/462 2012-02-09T16:16:58Z 2012-08-14T13:13:17Z <p>I can't monitor ParallelTable: </p> <pre><code>Monitor[ParallelTable[Pause; i, {i, 1, 10}], i] </code></pre> <p>just displays <code>i</code> until it is finished.</p> <p>Do you guys know of alternatives?</p> https://mathematica.stackexchange.com/q/148489 35 Good references for parallel programming in Mathematica Gaius https://mathematica.stackexchange.com/users/48373 2017-06-16T10:25:31Z 2017-09-30T08:50:37Z <p>There are plenty of good references for all sorts of Mathematica programming:</p> <ul> <li>This forum is a great example for specific questions/"getting things done".</li> <li>Books like those by R.Maeder (esp. the now ancient "Programming in Mathematica"), Ruskeepa, Tam, Trott, Wagner, Wellin, etc are all excellent at presenting stuff in a more organized, thorough fashion.</li> <li>The built-in WRI documentation can be quite good (but that's not the case here, in my opinion).</li> </ul> <p>Most of these books were published before parallel tools were a prominent feature in Mathematica. </p> <p>While Stackexchange answers are great for getting specific things done quickly (same goes for the Mathematica Cookbook), it's not in the spirit of this site to provide a bottom-up, textbook-like explanation of a relatively broad subject as this is.</p> <p>There are several lists of Mathematica-related references and links - most notably here <a href="https://mathematica.stackexchange.com/questions/18/where-can-i-find-examples-of-good-mathematica-programming-practice">18</a>, but the few parallel computation topics therein are somewhat specific/narrow.</p> <p>So the question is: do you know of books, or other well structured, thorough bottom-up discussions of parallel programming in Mathematica? Care to share your recommendations?</p> https://mathematica.stackexchange.com/q/313 32 How to collect result continuously (interruptible calculation) when running parallel calculations? Szabolcs https://mathematica.stackexchange.com/users/12 2012-01-19T22:41:14Z 2012-05-24T22:08:58Z <p>This is the most common pattern to compute a table of results:</p> <pre><code>Table[function[p], {p, parameters}] </code></pre> <p>(regardless of how it's implemented, it could be a <code>Map</code>)</p> <p>The problem with this is that if the calculation is interrupted before it's finished, the partial results will be lost.</p> <p>We can do this in a safely interruptible way like so:</p> <pre><code>Do[AppendTo[results, {p, function[p]}], {p, parameters}] </code></pre> <p>If this calculation is interrupted before it's finished, the intermediate results are still preserved. We can easily restart the calculation later, for those parameter values only for which <code>function[]</code> hasn't been run yet.</p> <p><strong>Question:</strong> What is the best way to achieve this when running calculations in parallel?</p> <p>Assume that <code>function[]</code> is expensive to calculate and that the calculation time may be different for different parameter values. The parallel jobs must be submitted in a way to make best use of the CPU. The result collection must not be shared between the parallel kernels as it may be a very large variable (i.e. I don't want as many copies of it in memory as there are kernels)</p> <hr> <p><strong>Motivation:</strong> I need this because I want to be able to make my calculations time constrained. I want to run the function for as many values as possible during the night. In the morning I want to stop it and see what I got, and decide whether to continue or not.</p> <hr> <p><strong>Notes:</strong></p> <p>I'm sure people will mention that <code>AppendTo</code> is inefficient and is best avoided in a loop. I think this is not an issue here (considering that the calculations run on the subkernels and <code>function[]</code> is expensive). It was just the simplest way to illustrate the problem. There could be other ways to collect results, e.g. using a linked list, and flattening it out later. <code>Sow</code>/<code>Reap</code> is not applicable here because they don't make it possible to interrupt the calculation.</p> <p>About the long running time: The most expensive part of the calculations I'm running are in C++ and called through LibraryLink, but they still take a very long time to finish.</p> https://mathematica.stackexchange.com/q/1116 30 Is it safe to launch/close kernels in the middle of a parallel calculation? Szabolcs https://mathematica.stackexchange.com/users/12 2012-02-01T17:30:10Z 2018-07-25T22:24:56Z <p>It appears that it is possible to launch additional kernels (or close existing ones) <em>during</em> a parallel calculation. The newly launched kernels will be utilized for the rest of the calculation.</p> <p>Here's a simple way to test and illustrate this (in a fresh kernel):</p> <pre><code>LaunchKernels Parallelize[ Table[Pause@RandomInteger; $KernelID, {i, 8}], Method -&gt; "FinestGrained" ] </code></pre> <p>Now use <code>Evaluation -&gt; Interrupt Evaluation...</code> to interrupt the evaluation and go into a subsession. In the subsession evaluate</p> <pre><code>$KernelCount (* check that we only have 2 kernels *) LaunchKernels (* launch 4 more *) Return (* return from the dialog *) </code></pre> <p>Notice that when the <code>Table[]</code> finally finishes, we get results between 1 and 6 (I got <code>{2, 1, 6, 5, 4, 3, 2, 1}</code>). This means that all 6 kernels have been utilized during the rest of the evaluation, even though some kernels were only launched in the middle of the calculation.</p> <p>It is also possible to close some kernels during the calculation. In another test I got the following messages ...</p> <p><img src="https://i.stack.imgur.com/ePKP1.png" alt="Mathematica graphics"></p> <p>... and the calculation seemed to finish correctly.</p> <p><strong>Question:</strong> Is it safe to do this? Can this break anything if the timings are wrong?</p> <hr> <p>Consider some code like this:</p> <pre><code>(* This uses LaunchKernels[] to launch more kernels as compute resources become available: *) manageKernels[] := ... (* ensure that manageKernels[] is always evaluated on the master kernel *) SetSharedFunction[manageKernels] Parallelize[ Table[manageKernels[]; compute[i], {i, 100}], Method -&gt; "FinestGrained" ] </code></pre> <p>Is this going to be safe? What if <code>manageKernels[]</code> uses <code>CloseKernels[]</code> as well?</p> https://mathematica.stackexchange.com/q/1883 27 List of Parallelize[]-able functions faleichik https://mathematica.stackexchange.com/users/219 2012-02-16T20:12:22Z 2012-02-16T20:59:30Z <p>The question is clear and is inspired by <a href="https://mathematica.stackexchange.com/q/1096/219">this question</a> and the first comment on <a href="https://mathematica.stackexchange.com/q/1877/219">this question</a>. Although there are many tricks for parallelizing things that are not <code>Parallelize[]</code>-able it would be handy to know beforehand whether the function you need is boostable with <code>Parallelize</code>.</p> <p>From the <a href="http://reference.wolfram.com/mathematica/ref/Parallelize.html?q=Parallelize&amp;lang=en" rel="nofollow noreferrer">official documentation</a> we have the following list:</p> <ul> <li>All listable functions with one argument.</li> <li>Structure-preserving functions: <code>Map</code>, <code>MapThread</code>, <code>MapIndexed</code>, <code>Scan</code>, <code>Apply</code>.</li> <li>Reductions: <code>Count</code>, <code>MemberQ</code>, <code>FreeQ</code>. </li> <li>Products: <code>Inner</code>, <code>Outer</code>, <code>Dot</code>.</li> <li>Iterators: <code>Table</code>, <code>Array</code>, <code>Product</code>, <code>Sum</code>.</li> </ul> <p>Does the complete list exist?</p> https://mathematica.stackexchange.com/q/5484 23 Does parallel programming use up large quantities of memory in Mathematica? Todd Allen https://mathematica.stackexchange.com/users/249 2012-05-13T01:27:47Z 2013-11-27T20:45:16Z <p>When I start Mathematica with a fresh kernel and load a program I am developing to analyze biology data, the amount of used system memory is 1.9 GB (free memory is 14.5 GB). These memory values are reported by an application external to Mathematica. </p> <p>After I run the code I have developed, the amount of used system memory is 13.2 GB (free memory is 3.2 GB). So, this means that running my code has consumed 13.2-1.9 = 11.3 GB of memory. After running the code, <code>MaxMemoryUsed[]</code> reports a value of 2.7 GB.</p> <ol> <li><p>Even though <em>Mathematica</em> is reporting <code>MaxMemoryUsed</code> at 2.7Gb, my system is still reporting 13.2 GB of used memory. If this is true, why hasn't Mathematica cleared the 11.3 - 2.7 = 8.6 GB of memory it says it hasn't used and returned it to the system?</p></li> <li><p>I have tried setting <code>$HistoryLength = 0</code> and it has not made a difference in the reported memory usage.</p></li> <li><p>My code (posted below) does use several <code>ParallelTable</code> statements. Do these statements send a copy of the memory content to each parallel kernel?</p></li> <li><p>Do you have suggestions to solve this mystery of where is my memory?</p></li> </ol> <p>Thank you for your thoughts. Todd</p> <p>--------------- Mathematica Code ------------</p> <pre><code>(* load cel data *) time0 = AbsoluteTime[]; SetDirectory[cellocation]; celfilenames = FileNames[]; (* names of data files to import *) celvarnames = Table[StringSplit[celfilenames[[i]], {"."}][], {i, 1, Length[celfilenames]}] ;(* names of datafiles, with extensions \ removed, to be used downstream *) Table[microarray[celvarnames[[i]]] = Import[celfilenames[[i]]], {i, 1, Length[celfilenames]}];(* assign data to microarray variable \ using celvarnames as indexes to microarray; each chip's data is \ assigned to a different index to microarray *) chipsize = chipdimensions[ microarray[celvarnames[]]] (* determine chip dimensions *) (* load affy cdf data *) SetDirectory[affycdflocation]; cdffilenames = FileNames[]; cdffile = Import[Flatten[StringCases[cdffilenames, ___ ~~ ".cdf" ~~ ___]][[ 1]]]; (* convert hybidization symbol names to string names *) experimentchips = Map[ToString, experimentchips]; controlchips = Map[ToString, controlchips]; Print["Experimental condition chips: ", experimentchips] Print[] Print["Control condition chips: ", controlchips] pmindexes = Table[Select[cdffile[[i, 3, 2]], #[] != #[] &amp;][[All, 4]], {i, 1, Length[ cdffile]}]; (* perfect match probe indexes by probeset; same \ order as affy cdf file *) mmapmindices = Map[Transpose, Thread[affyindextoMMAindices[pmindexes, chipsize]]]; (* create Mathematica indices equivalent to Affy pm \ index positions by probeset *) pmtemp = ParallelTable[ Extract[microarray[celvarnames[[i]]], mmapmindices[[j]]], {i, 1, Length[celvarnames]}, {j, 1, Length[mmapmindices]}]; (* get pm data from all the chips; data \ grouped by chip; within each chip,data is organized in same order as \ probes listed in cdf file *) Table[pmsignal[celvarnames[[i]]] = pmtemp[[i]], {i, 1, Length[celvarnames]}] ;(* put the pm data into the pmsignal \ variable, using celvarnames as indexes referring to the different \ chips *) Clear[pmtemp, microarray]; (* no need to retain duplicate information in memory, \ once data is assigned to specific chips *) Table[pmstdev[celvarnames[[i]]] = Map[StandardDeviation, pmsignal[celvarnames[[i]]]], {i, 1, Length[celvarnames]}]; (* compute standard deviations by pm probeset,for each chip, and \ store it in pmstdev using celvarnames as indexes *) subsetsize = 0.10*Length[ pmstdev[celvarnames[[ 1]]]]; (* Use 10% of probesets to calculate clusters below *) Table[pmstdevsubset[celvarnames[[i]]] = BlockRandom[SeedRandom; RandomChoice[pmstdev[celvarnames[[i]]], Floor[subsetsize]]] , {i, 1, Length[celvarnames]}]; (* select a random subset of pm probset stdev for establishing \ "normal" pm probeset standard deviations for each chip *) pmtemp = ParallelTable[ Sort[FindClusters[pmstdevsubset[celvarnames[[i]]], 3, DistanceFunction -&gt; EuclideanDistance, Method -&gt; "Agglomerate"]][[-1]], {i, 1, Length[celvarnames]}]; (* Find the largest cluster for each pm \ standard deviation subset by chip; the mean of these clusters will \ set the "normal" pm variation for each chip; only find 3 clusters to \ make calculation quicker *) Table[pmstdevcluster[celvarnames[[i]]] = pmtemp[[i]], {i, 1, Length[pmtemp]}] ;(* assign pmtemp results to pmstdevcluster using \ celvarnames as keys *) Clear[pmtemp]; (* no need to store same data twice *) Table[pmpostocheck[celvarnames[[i]]] = Flatten[Position[pmstdev[celvarnames[[i]]], x_ /; x &gt; Mean[pmstdevcluster[celvarnames[[i]]]]]], {i, 1, Length[celvarnames]}]; (* determine each chips pm probeset positions whose st devs are \ greater than the "normal" threshold, which is calculated by taking \ the mean of the largest clusters for each chip - stored in \ pmstdevcluster *) pmexperimentemp = ParallelTable[ Flatten[Drop[ errorcorrection[pmsignal, experimentchips[[i]], pmpostocheck], 1], 2], {i, 1, Length[experimentchips]}];(*calculate improved pmsignal in \ experimental chip probesets that have unusually high signal variation \ using the same probesets in comparable experimental condition chips \ as a surrogate*) pmcontroltemp = ParallelTable[ Flatten[Drop[ errorcorrection[pmsignal, controlchips[[i]], pmpostocheck], 1], 2], {i, 1, Length[controlchips]}];(*calculate improved pmsignal in control \ chip probesets that have unusually high signal variation using the \ same probesets in comparable control condition chips as a surrogate*) Table[pmsignal[experimentchips[[i]]] = ReplacePart[pmsignal[experimentchips[[i]]], Thread[pmpostocheck[experimentchips[[i]]] -&gt; pmexperimentemp[[i]]]], {i, 1, Length[experimentchips]}]; (* replace spurious pmsignal with \ replacement values in experimental condition chips *) Table[pmsignal[controlchips[[i]]] = ReplacePart[pmsignal[controlchips[[i]]], Thread[pmpostocheck[controlchips[[i]]] -&gt; pmcontroltemp[[i]]]], {i, 1, Length[controlchips]}]; (* replace spurious pmsignal with \ replacement values in control condition chips *) Clear[pmexperimentemp, pmcontroltemp]; (* don't store data longer than needed *) Table[experimentchippmmean[experimentchips[[i]]] = Mean[Flatten[pmsignal[experimentchips[[i]]]]], {i, 1, Length[experimentchips]}]; (* calculate each experimental chips pmsignal mean to use in \ standardization *) Table[controlchippmmean[controlchips[[i]]] = Mean[Flatten[pmsignal[controlchips[[i]]]]], {i, 1, Length[controlchips]}]; (* calculate each control chips pmsignal \ mean to use in standardization *) Table[experimentchippmstdev[experimentchips[[i]]] = StandardDeviation[Flatten[pmsignal[experimentchips[[i]]]]], {i, 1, Length[experimentchips]}]; (* calculate each experimental chips \ pmsignal standard deviation to use in standardization *) Table[controlchippmstdev[controlchips[[i]]] = StandardDeviation[Flatten[pmsignal[controlchips[[i]]]]], {i, 1, Length[controlchips]}]; (* calculate each control chips pmsignal \ standard deviation to use in standardization *) pmexpstandtemp = ParallelTable[ Map[(# - experimentchippmmean[experimentchips[[i]]])/ experimentchippmstdev[experimentchips[[i]]] &amp;, pmsignal[experimentchips[[i]]]], {i, 1, Length[experimentchips]}];(* standardize each experimental chips \ pmsignal *) pmcontrolstandtemp = ParallelTable[ Map[(# - controlchippmmean[controlchips[[i]]])/ controlchippmstdev[controlchips[[i]]] &amp;, pmsignal[controlchips[[i]]]], {i, 1, Length[controlchips]}];(* standardize each control chips pmsignal \ *) Table[experimentstandard[experimentchips[[i]]] = pmexpstandtemp[[i]], {i, 1, Length[experimentchips]}]; (* reassign data from pmexpstandtemp to proper indexed variable *) Table[controlstandard[controlchips[[i]]] = pmcontrolstandtemp[[i]], {i, 1, Length[controlchips]}];(* reassign data from pmcontrolstandtemp to \ proper indexed variable *) Clear[pmexpstandtemp, pmcontrolstandtemp]; (* free up memory *) time1 = AbsoluteTime[] - time0 </code></pre> https://mathematica.stackexchange.com/q/131573 22 Restarting Mathematica automatically Vadim https://mathematica.stackexchange.com/users/44677 2016-11-19T18:43:59Z 2018-03-20T10:09:28Z <p>I run a large computation in a loop. At each cycle of the loop I produce large arrays that I save on a disk. At each cycle I would like to <code>Quit</code> <em>Mathematica</em> and then restart automatically, loading necessary files from the disk. I need this to manage memory usage. I tried various tricks but I do not think anything short of <code>Quit</code> will work in my case. Please let me know if there is a way to do it. I do not want to use one of the kernels to control the <code>Quit</code> process because I want to use all kernels for parallel computation and clear up memory everywhere at each cycle. Thank you.</p> https://mathematica.stackexchange.com/q/82496 22 Two, maybe related, issues with ParallelTable Fred Simons https://mathematica.stackexchange.com/users/20253 2015-05-03T12:04:28Z 2016-01-13T01:25:16Z <p>I will use a very simple example for demonstrating two issues I face with ParallelTable, one with respect to the speed and one with respect to the use of memory. These issues might be related.</p> <p>Let us start with the speed.</p> <pre><code>args=RandomReal[1, {10^7}]; Table[x, {x, args}]; // Timing (* {0.312002,Null} *) ParallelTable[x, {x, args}]; // Timing (* {8.40845,Null} *) </code></pre> <p>This is surprising. I used Timing instead of AbsoluteTiming, so the result shows that the master kernel needs 8 seconds for handling the results of my 4 subkernels. What is it doing all the time?</p> <p>This question has been asked before, see <a href="https://mathematica.stackexchange.com/questions/28445/paralleltable-70-times-slower-on-16-cores-than-table-on-single-core">here</a>. There is a reference to a great answer of @OleksandrR (see <a href="https://mathematica.stackexchange.com/questions/2886/transferring-a-large-amount-of-data-in-parallel-calculations/2919#2919">here</a>) to another question, in which he showed that the bottleneck is a call to MemberQ with second argument$Aborted, which in turn forces Mathematica to do a lot of unpacking packed arrays.</p> <p>We can test this in our example by adding a definition to MemberQ that never actually will be used, it only prints a message when MemberQ is called. One should never modify built in functions, but this seems to be pretty safe.</p> <pre><code>On["Packing"]; On[General::stop]; Unprotect[MemberQ];Clear[MemberQ]; MemberQ[_, $Aborted] /; (Print["MemberQ[arg,$Aborted] called."]; False) := Null; ParallelTable[x, {x, args}]; Unprotect[MemberQ]; Clear[MemberQ]; (* DeveloperFromPackedArray::punpack1: Unpacking array with dimensions {544}. &gt;&gt; DeveloperFromPackedArray::punpackl1: Unpacking array with dimensions {544,2} to level 1. &gt;&gt; DeveloperFromPackedArray::punpack1: Unpacking array with dimensions {2}. &gt;&gt; DeveloperFromPackedArray::punpack1: Unpacking array with dimensions {2}. &gt;&gt; General::stop: Further output of DeveloperFromPackedArray::punpack1 will be suppressed during this calculation. &gt;&gt; MemberQ[arg, $Aborted] called. *) </code></pre> <p>When I run this command with Off[General::stop], I see about 650 unpacking array messages. So it might very well be that all this unpacking is responsible for the 8 seconds master kernel time.</p> <p>However, what we see here is not quite in accordance with OleksandrR's statement. All the unpacking is done before <code>MemberQ[_,$Aborted]</code> is called. That explains why the following dirty trick: simply redefine <code>MemberQ[_, $Aborted]</code> as False, so that no unpacking would be needed, does not work. </p> <pre><code>On["Packing"]; On[General::stop]; Unprotect[MemberQ];Clear[MemberQ]; MemberQ[_,$Aborted] = False; ParallelTable[x, {x, args}]; Unprotect[MemberQ]; Clear[MemberQ]; (* DeveloperFromPackedArray::punpack1: Unpacking array with dimensions {544}. &gt;&gt; ... *) </code></pre> <p>For the same reason, @Szabolcs solution with a locally redefined function MemberQ does not work either.</p> <p>In my example, there is a lot of unpacking, but not because of a call to MemberQ. Unfortunately, Mathematica does not tell me which function is initiating the unpacking.</p> <p>There is another issue with this example, maybe related. Start with a fresh kernel and obeserve the amount of used memory.</p> <pre><code>Dynamic[MemoryInUse[], UpdateInterval-&gt;1] args=RandomReal[1, {10^7}]; </code></pre> <p>On my computer, the amount of memory increases from 22 MB to 103 MB, about as expected.</p> <pre><code>Table[x, {x, args}]; </code></pre> <p>No change in the amount of used memory, as expected.</p> <pre><code>ParallelTable[x, {x, args}]; </code></pre> <p>During the computation, the amount of used memory goes up to at least 584 MB, and ends with 304 MB. So this computation costs permanently 200 MB of memory. Each time when we execute the ParallelTable command we loose 200 MB of memory. That looks like a bug to me; just as with Table I would have expected no change in the amount of memory in use.</p> <p>Also the following command gives an increase of 200 MB of used memory:</p> <pre><code>ParallelTable[x, {x, args}, Method-&gt;"CoarsestGrained"]; </code></pre> <p>And with the following command, the kernel silently crashes:</p> <pre><code>ParallelTable[x, {x, args}, Method-&gt;"FinestGrained"]; </code></pre> <p>I am a little bit surprised that such an elementary function as ParallelTable shows such strange behaviour. Any comment is highly welcomed.</p> https://mathematica.stackexchange.com/q/10969 20 Parallelization problem in LinearSolve and Minimize mak maak https://mathematica.stackexchange.com/users/2376 2012-09-23T03:35:29Z 2012-09-27T04:15:34Z <p>My CPU has got 8 cores (it is Intel Core i7-2600 3.40 GHz). When I try to solve a linear matrix equation using LinearSolve for large matrices, Mathematica just uses 4 cores to solve the problem (CPU usage will be 50%). So it means that there is a problem in the Parallel computation options. When I try to Minimize a huge function with several variables, it is even worse and Mathematica just uses one core (CPU usage is about 12%)!!</p> <p>I am not very familiar with Parallel computation, so will be very appreciative if you can help me to solve my problem. How can I use all capacity of the CPU when I run LinearSolve for very large matrices and Minimize for huge functions and make CPU usage 100% to make the computation time as short as possible on my machine??</p> <p>Thank you very much.</p> <p>**</p> <h2>Edit 1:</h2> <p>**</p> <p>My computer for this analysis:</p> <pre><code>t = AbsoluteTime[]; primelist = Table[Prime[k], {k, 1, 20000000}]; time2 = AbsoluteTime[] - t </code></pre> <p>Yields a load of 12% on my CPU and time2=43.37 and by breaking this analysis into 8 cores:</p> <pre><code>t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 2500000}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 2500001, 5000000}]]; job3 = ParallelSubmit[Table[Prime[k], {k, 5000001, 7500000}]]; job4 = ParallelSubmit[Table[Prime[k], {k, 7500001, 10000000}]]; job5 = ParallelSubmit[Table[Prime[k], {k, 10000001, 12500000}]]; job6 = ParallelSubmit[Table[Prime[k], {k, 12500001, 15000000}]]; job7 = ParallelSubmit[Table[Prime[k], {k, 15000001, 17500000}]]; job8 = ParallelSubmit[Table[Prime[k], {k, 17500001, 20000000}]]; {a1, a2, a3, a4, a5, a6, a7, a8} = WaitAll[{job1, job2, job3, job4, job5, job6, job7, job8}]; time2 = AbsoluteTime[] - t </code></pre> <p>Yields 100% load on CPU and time2=17.16</p> <p>**</p> <h2>Edit 2:</h2> <p>**</p> <p>To make it completely clear what is happening on my computer and what is my problem, please have a look at the following examples:</p> <p>First if I want to check number of processors and kernels on my machine:</p> <pre><code>$ProcessorCount$KernelCount </code></pre> <p>The results are 4 and 8 respectively on my machine.</p> <p>Now if I want to see MKL conditions on my machine and also how much "CPU usage" reported by the system monitor correspond to actual performance, I can run this in Mathematica:</p> <pre><code>Clear["Global*"]; a = RandomReal[{1, 2}, {20000, 20000}]; b = RandomReal[{1}, {20000}]; Table[SetSystemOptions["MKLThreads" -&gt; i]; Print["Case=", i]; Print[SystemOptions["MKLThreads"]]; t = AbsoluteTime[]; LinearSolve[a, b]; time2 = AbsoluteTime[] - t; Print["t(", i, ")=", time2]; Print["******"], {i, 4}]; </code></pre> <p>You can see the result including number of MKL threads and computation time for each case below:</p> <pre><code>Case=1 {MKLThreads-&gt;1} t(1)=202.9560000 ****** Case=2 {MKLThreads-&gt;2} t(2)=120.3696000 ****** Case=3 {MKLThreads-&gt;3} t(3)=93.5532000 ****** Case=4 {MKLThreads-&gt;4} t(4)=88.5300000 ****** </code></pre> <p>While the CPU usage for Case1=12%, Case2=25%, Case3=37% and Case4=50%, reported by the system monitor. You can see that in this case "CPU usage" reported by the system monitor correspond to actual performance and the more CPU usage we observe, the less computation time we have.</p> <p>Now if I increase the number of MKLThreads in SetSystemOptions["MKLThreads" -> ?] to values more than 4 (I mean 5 to 8), I can see that it doesn't have any effect on compuation time and CPU usage. The same thing happens if I change the number of ParallelThreadNumber in SetSystemOptions["ParallelOptions" -> {"ParallelThreadNumber" -> ?}], means that the computation time and CPU usage in this case do not depend on the ParallelThreadNumber. You can see the cases below:</p> <pre><code>SetSystemOptions["ParallelOptions" -&gt; {"ParallelThreadNumber" -&gt; 1}]; Print[SystemOptions["ParallelOptions" -&gt; "ParallelThreadNumber"]]; SetSystemOptions["MKLThreads" -&gt; 8]; Print[SystemOptions["MKLThreads"]]; t=AbsoluteTime[]; LinearSolve[a, b]; time2=AbsoluteTime[] - t; Print["t=", time2]; </code></pre> <p>The result is (CPU usage=50% during analysis):</p> <pre><code>{ParallelOptions-&gt;{ParallelThreadNumber-&gt;1}} {MKLThreads-&gt;4} t=85.3008000 </code></pre> <p>And for other case:</p> <pre><code>SetSystemOptions["ParallelOptions" -&gt; {"ParallelThreadNumber" -&gt; 8}]; Print[SystemOptions["ParallelOptions" -&gt; "ParallelThreadNumber"]]; SetSystemOptions["MKLThreads" -&gt; 8]; Print[SystemOptions["MKLThreads"]]; t=AbsoluteTime[]; LinearSolve[a, b]; time2=AbsoluteTime[] - t; Print["t=", time2]; </code></pre> <p>The result is (Again CPU usage=50% during analysis):</p> <pre><code>{ParallelOptions-&gt;{ParallelThreadNumber-&gt;8}} {MKLThreads-&gt;4} t=85.3476000 </code></pre> <p>As you can see, the CPU usage and computation time do not change when I increase MKLThreads to more than 4 (e.g 5 to 8) and they are also independent of the ParallelThreadNumber.</p> <p>Another interesting example is about the case I mentioned in edit 1. Please have a look at these examples and results and CPU usage for each case:</p> <p>1)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; primelist = Table[Prime[k], {k, 1, 20000000}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=43.37 and CPU usage=12% </p> <p>2)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 10000000}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 10000001, 20000000}]]; {a1, a2} = WaitAll[{job1, job2}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=30.01 and CPU usage=25% </p> <p>3)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 6666666}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 6666667, 13333332}]]; job3 = ParallelSubmit[Table[Prime[k], {k, 13333333, 20000000}]]; {a1, a2, a3} = WaitAll[{job1, job2, job3}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=23.46 and CPU usage=37% </p> <p>4)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 5000000}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 5000001, 10000000}]]; job3 = ParallelSubmit[Table[Prime[k], {k, 10000000, 15000000}]]; job4 = ParallelSubmit[Table[Prime[k], {k, 15000001, 20000000}]]; {a1, a2, a3, a4} = WaitAll[{job1, job2, job3, job4}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=21.52 and CPU usage=50% </p> <p>5)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 3333333}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 3333334, 6666666}]]; job3 = ParallelSubmit[Table[Prime[k], {k, 6666667, 9999999}]]; job4 = ParallelSubmit[Table[Prime[k], {k, 10000000, 13333333}]]; job5 = ParallelSubmit[Table[Prime[k], {k, 13333334, 16666666}]]; job6 = ParallelSubmit[Table[Prime[k], {k, 16666667, 20000000}]]; {a1, a2, a3, a4, a5, a6} = WaitAll[{job1, job2, job3, job4, job5, job6}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=18.28 and CPU usage=75% </p> <p>6)</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[Table[Prime[k], {k, 1, 2500000}]]; job2 = ParallelSubmit[Table[Prime[k], {k, 2500001, 5000000}]]; job3 = ParallelSubmit[Table[Prime[k], {k, 5000001, 7500000}]]; job4 = ParallelSubmit[Table[Prime[k], {k, 7500001, 10000000}]]; job5 = ParallelSubmit[Table[Prime[k], {k, 10000001, 12500000}]]; job6 = ParallelSubmit[Table[Prime[k], {k, 12500001, 15000000}]]; job7 = ParallelSubmit[Table[Prime[k], {k, 15000001, 17500000}]]; job8 = ParallelSubmit[Table[Prime[k], {k, 17500001, 20000000}]]; {a1, a2, a3, a4, a5, a6, a7, a8} = WaitAll[{job1, job2, job3, job4, job5, job6, job7, job8}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=17.16 and CPU usage=100%</p> <p>But if I make this analysis using ParallelTable, interestingly the CPU usage is 100%, but computation time is 45.81!!! It means that computation time is quite the same with number 1 when I do this analysis with Table on one core (CPU usage=12%)!!</p> <pre><code>t = AbsoluteTime[]; primelist = ParallelTable[Prime[k], {k, 1, 20000000}]; time2 = AbsoluteTime[] - t </code></pre> <p>Result: time2=45.81 and CPU usage=100%</p> <p>I also checked NMinimize for my big function with 75 (or more variables) using all methods available in Mathematica including Automatic, DifferentialEvolution, NelderMead, RandomSearch, and SimulatedAnnealing. The computation time for all of them is quite the same and CPU usage for all methods is only 12%. So it looks that minimization method I use in NMinimize cannot change parallelization conditions.</p> <p>Now I think my conditions and problems are completely clear, so I would be very appreciative if someone can help me to use all capacity of my CPU in LinearSolve and NMinimize (or Minimize). I still wonder how I can make CPU usage in these cases 100%. In this way we can check whether CPU usage corresponds to actual perfomanse (like what we could see in the examples mentioned above) for LinearSolve and NMinimize or not??</p> <p>Thank you very much.</p> <p>**</p> <h2>Edit 3</h2> <p>**</p> <p>The function I am trying to minimize is a large function including many variables. The general format of the function is sth like this:</p> <pre><code>(15 (-2.14286*10^-8 Log[E^(-1.*10^6 phi2) + E^(1.*10^6 phi2)] + uu1))^2 + 225 (2.14286*10^-8 Log[E^(-1.*10^6 phi2) + E^(1.*10^6 phi2)] -2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] -2 uu1 + uu1)^2 + 225 (2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] -2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] + uu1 - 2 uu1 + uu1)^2 + 225 (2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] - 2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] + uu1 - 2 uu1 + uu1)^2 + 225 (-2.14286*10^-8 Log[E^(-1.*10^6 phi2) + E^(1.*10^6 phi2)]+ 2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] + uu1 - 2 uu1)^2 + 225 (2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] - 2.14286*10^-8 Log[E^(-1.*10^6 (-phi2 + phi2)) + E^(1.*10^6 (-phi2 + phi2))] + uu1 - 2 uu1 + uu1)^2 + ((15 (2.14286*10^-8 Log[E^(-1.*10^6 phi2) + E^(1.*10^6 phi2)] + uu1))^2) + (0.00918367 phi2 + (-0.00175179 - 11/112 (0.007848)) (1 - (1.*10^12 (phi2)^2)/(Log[E^(-1.*10^6 phi2) + E^(1.*10^6 phi2)])^2) + ... </code></pre> <p>Where uu1[i], uu3[i] and phi2[i] are variables. The issue is that the number of variables can increase to a large number (for example 5000 or even more) which make the function hugely big!! So if I cannot use all capacity of the CPU it takes maybe days to minimize such a function, even though one computer with full capacity of the CPU is not enough to solve such a problem too, but the first step is to learn how to configure parallelization for NMinimize (or FindRoot) on a single machine to be able to extend it to parallelization on several remote machines.</p> <h2>Edit 4:</h2> <p>An example of the complete form of the function with 75 variables is:</p> <p><a href="http://pastebin.com/7T6r4ckK">Pastebin link</a></p> <p>Where variables (unknown parameters) are:</p> <pre><code>{uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, uu1, uu3, phi2, P1, F1, M1, PN, FN, MN} </code></pre> <p>I know this function is extremely instable, but the optimum point of the function is also known which is equal to zero, so I am trying to find the values of the parameters which make the function minimum (zero). The parameters which can make the whole function as small as possible are the best answers.</p> <p>**</p> <h2>Edit 5</h2> <p>**</p> <p>Thank you all guys for your helpful comments. Based on what KAI and Oleksandr R. mentioned, and as far as I could understand, LinearSolve uses all capacity of the CPU cores to solve the equation involving large matrices. Consequently, it seems that if I want to solve some linear matrix equations for a few times, the best method is to solve each of them one by one in a LOOP to make it most efficient. In this way Mathemathica is able to use all capacity of CPU cores in each step and solve the equation in the most efficient way (in each step) and goes to the next step. But if you have a look at these 2 examples, apparently it is not like this and Parallelization forces Mathematica to solve the problem involving LinearSolve in a way that is likely more efficient and faster. You can check these examples on your computer. Based on the comments we had here, I am wondering how we can explain these examples.</p> <p>Example 1:</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; NN = 8; CC = Array[cc, NN]; For[i = 1, i &lt; (NN + 1), i++, Clear[a, b]; a = RandomReal[{i, i + 1}, {6000, 6000}]; b = RandomReal[{i}, {6000}]; CC[[i]] = LinearSolve[a, b]; ]; time2 = AbsoluteTime[] - t </code></pre> <p>For example 1 CPU usage is 50% and time2=23.4</p> <p>Example 2:</p> <pre><code>Clear["Global*"]; t = AbsoluteTime[]; job1 = ParallelSubmit[a1 = RandomReal[{1, 2}, {6000, 6000}]; b1 = RandomReal[{1}, {6000}]; c1 = LinearSolve[a1, b1]]; job2 = ParallelSubmit[a2 = RandomReal[{2, 3}, {6000, 6000}]; b2 = RandomReal[{2}, {6000}]; c2 = LinearSolve[a2, b2]]; job3 = ParallelSubmit[a3 = RandomReal[{3, 4}, {6000, 6000}]; b3 = RandomReal[{3}, {6000}]; c3 = LinearSolve[a3, b3]]; job4 = ParallelSubmit[a4 = RandomReal[{4, 5}, {6000, 6000}]; b4 = RandomReal[{4}, {6000}]; c4 = LinearSolve[a4, b4]]; job5 = ParallelSubmit[a5 = RandomReal[{5, 6}, {6000, 6000}]; b5 = RandomReal[{5}, {6000}]; c5 = LinearSolve[a5, b5]]; job6 = ParallelSubmit[a6 = RandomReal[{6, 7}, {6000, 6000}]; b6 = RandomReal[{6}, {6000}]; c6 = LinearSolve[a6, b6]]; job7 = ParallelSubmit[a7 = RandomReal[{7, 8}, {6000, 6000}]; b7 = RandomReal[{7}, {6000}]; c7 = LinearSolve[a7, b7]]; job8 = ParallelSubmit[a8 = RandomReal[{8, 9}, {6000, 6000}]; b8 = RandomReal[{8}, {6000}]; c8 = LinearSolve[a8, b8]]; {R1, R2, R3, R4, R5, R6, R7, R8} = WaitAll[{job1, job2, job3, job4, job5, job6, job7, job8}]; time2 = AbsoluteTime[] - t </code></pre> <p>For example 2 CPU usage=100% and time2=19.8</p> https://mathematica.stackexchange.com/q/55914 19 Partial Differential Equation in Parallel user18790 https://mathematica.stackexchange.com/users/18790 2014-07-26T11:16:04Z 2019-10-07T07:32:39Z <p>is there any native way to implement multi-core parallel solving of PDE in Wolfram Mathematica? </p> <p>WM 10 now supports Finite Elements Method, but it is actually useless without parallelization. Usually PDE-systems are heavy and no one solves it using just one core. If there is any sort of domain decomposition or some other way parallelize the calculation?</p> https://mathematica.stackexchange.com/q/103548 19 Is it possible to Parallelize Select? jjstankowicz https://mathematica.stackexchange.com/users/6506 2016-01-07T07:09:18Z 2018-07-21T03:47:11Z <p>Given a large list of elements, is it possible to improve <code>Select</code> by parallelizing?</p> <p>An example: from a 10,000,000-element list of integers between 1 and 10, select all primes</p> <pre><code>rl = RandomInteger[10, {10^7}]; Select[rl, PrimeQ] // AbsoluteTiming // First (* 4.18468 *) rlp = Partition[rl, 4]; LaunchKernels[]; Union[ParallelMap[Select[#, PrimeQ] &amp;, rlp]]] // AbsoluteTiming // First (* 83.7938 *) </code></pre> <p>I demonstrated my first naive attempt.</p> <ul> <li>What causes the huge time increase? Passing huge lists between kernels?</li> <li>Is there a better way to do this? (One that works for lists of elements other than integers)</li> <li>A slight variant: if I generate the large list in the first place, is it faster to use <code>Reap</code>/<code>Sow</code>? (Which I see also has some parallelization issues.)</li> </ul> https://mathematica.stackexchange.com/q/123182 19 Status of all Evaluators? M.R. https://mathematica.stackexchange.com/users/403 2016-08-04T22:56:28Z 2016-10-04T15:32:37Z <p>Since I have 4 cores on my machine I typically have 4 notebooks open each with their own kernel evaluator. I'd like to construct a dynamic pane similar to the parallel kernels status window:</p> <p><a href="https://i.stack.imgur.com/DqWCk.png" rel="noreferrer"><img src="https://i.stack.imgur.com/DqWCk.png" alt="enter image description here"></a></p> <p>But that collects a few statistics of all them:</p> <ul> <li>evaluation queue size </li> <li>cpu/gpu memory in use over time</li> <li>virtual memory in use</li> <li>uptime </li> <li>status</li> </ul> <p>Here's how I get the names of the ones in use, but I'm unsure of how to query for further details:</p> <pre><code>n = Notebooks[] Union[CurrentValue[#, "Evaluator"] &amp; /@ n] </code></pre> <p><a href="https://i.stack.imgur.com/4iTYa.png" rel="noreferrer"><img src="https://i.stack.imgur.com/4iTYa.png" alt="enter image description here"></a></p> https://mathematica.stackexchange.com/q/20356 18 Mathematica Parellelization on HPC fpghost https://mathematica.stackexchange.com/users/1882 2013-02-27T13:51:25Z 2015-02-03T02:02:53Z <p>I'm currently using <em>Mathematica</em> on a High Performance Cluster (HPC) which consists of many compute nodes each with around 16 cores. I currently run my <em>Mathematica</em> script on 20 of the nodes that invokes 10 cores and 10 of the subkernels licenses in Parallel, meaning I use 20 Mathkernel licenses and 200 subkernel licenses. </p> <p>The problem is we have limited Mathkernel licenses (36, and for me to be using 20 of them is unfair on everyone else!) although ample subkernel licenses (288). Is there a way I can just use a single (or at least fewer) Mathkernel licenses to invoke the 200 subkernels I need? </p> <p>Currently in each of the 20 scripts I just have</p> <pre><code>LaunchKernels; ParellelTable[....]; </code></pre> <p>which launches the 10 local subkernels on each node, but could I specify different nodes to launch subkernels on perhaps? Thereby I would only need to launch one Mathkernel which could invoke the 200 subkernels spread across the compute nodes.</p> https://mathematica.stackexchange.com/q/612 17 Connecting to and disconnecting from a continuously running kernel, on demand Szabolcs https://mathematica.stackexchange.com/users/12 2012-01-24T14:19:01Z 2018-07-19T19:42:12Z <p>I realized that there are lots of advantages to driving C/C++/FORTRAN code directly from Mathematica as LibraryLink functions (instead of running them from the command line or a shell script, as I have usually done before). This will give access to a lot of functionality that is difficult or time consuming to implement in a low level language (<a href="https://mathematica.stackexchange.com/questions/216/adaptive-sampling-for-slow-to-compute-functions-in-2d">example</a>).</p> <p>There are disadvantages too, mainly because the running environment is typically a remote server, and not a local workstation (i.e. I don't get a notebook GUI). So, to get around some of the disadvantages,</p> <p><strong>Is the following feasible to implement (see below)?</strong></p> <p>Can we have a Mathematica master kernel running a remote kernel, running parallel calculations in subkernels, and do the following:</p> <p>While the calculation is running, connect to the master computer through MathLink from a laptop; check the state of the calculations, perhaps do some quick-to-compute preliminary analysis on the so far calculated results; make decisions about continuing the calculation or not; then disconnect. It should be possible to connect to and disconnect from the server as many times as necessary without aborting the calculations for good.</p> <p>Do you think that such a thing is (theoretically) possible to implement with the current version of Mathematica? Or perhaps the current features are not general enough to allow it?</p> <p>Before sitting down to study the documentation in detail and try to implement this, I was wondering if anyone is aware of any showstopper limitations (or if anyone has tried to implement it).</p> https://mathematica.stackexchange.com/q/84380 17 Does Mathematica take advantage of hyper-threading (if it is available)? Andrew https://mathematica.stackexchange.com/users/1185 2015-05-25T21:55:56Z 2015-05-26T21:00:30Z <p>I do not know much about computer architecture, but some Intel processors have <a href="http://en.wikipedia.org/wiki/Hyper-threading">hyper-threading technology</a>, which can apparently improve parallelization for applications that take advantage of it. In a processor with hyper-threading technology, the number of threads is usually twice the number of cores -- whereas in a processor <em>without</em> hyper-threading, the number of threads is equal to the number of cores.</p> <p>Does Mathematica take advantage of hyper-threading?</p> <p>My old (2007) desktop computer has an old Intel Core2 processor with 2 cores that does <em>not</em> support hyper-threading. So the processor has 2 threads and 2 cores. If I execute <code>$ProcessorCount</code> in Mathematica, it gives the output <code>2</code>.</p> <p>However, some new processors (i.e., Intel i7 series) support hyper-threading. For example, Intel's website says that the <a href="http://ark.intel.com/products/80806/">i7-4790</a> has 8 threads and 4 cores. If I were to execute <code>$ProcessorCount</code> on such a system, would the output be <code>4</code> or <code>8</code>?</p> https://mathematica.stackexchange.com/q/7017 17 Parallel compiled functions running on parallel sub-kernels halirutan https://mathematica.stackexchange.com/users/187 2012-06-19T04:53:29Z 2012-06-21T13:27:51Z <h1>Question</h1> <p>Why is it, that compiled, <code>Listable</code>, parallel functions which work perfectly fine on the main kernel, do not run in parallel on sub-kernels?</p> <h1>Details</h1> <h2>First Example</h2> <p>Let me give a first example. I compile a function $f:\mathbb{R}\to\mathbb{R}$ which is a simple sum of sine-functions with the options <code>CompilationTarget -&gt; "C"</code>, <code>RuntimeAttributes -&gt; {Listable}</code> and <code>Parallelization -&gt; True</code>. Due to the <code>Listable</code> attribute I can now call the function with tensor parameters and due to the parallelization, the values in the tensor are processed in parallel. If you are on a slow machine, adjust <code>n</code> to be not that high:</p> <pre><code>f = Compile[{{t, _Real, 0}}, #, CompilationTarget -&gt; "C", RuntimeAttributes -&gt; {Listable}, Parallelization -&gt; True] &amp;@ Sum[Sin[2.0 Pi k t]/k, {k, 1000}]; data = With[{n = 1000000}, Table[t, {t, 0, 1, 1/(n - 1)}]]; f[data]; </code></pre> <p>Looking at the system monitor during the calculation shows, that all processors are running at 100%</p> <p><img src="https://i.stack.imgur.com/6NUDc.png" alt="enter image description here"></p> <p>If you like you can compare the speed of this execution with for instance <code>f /@ data</code>.</p> <h2>Under the hood</h2> <p><em>This may be only correct for Linux and OS X!</em></p> <p>What happens when you use <code>Compile</code> with the "C" option is, that a shared library is created from your <em>Mathematica</em>-code which contains a function that can be called. The libraries are stored in a folder which is specific to the process id of a specific kernel. Let's make a short function, to print the important stuff of such a compiled function. I extract from a <code>CompiledFunction</code> the information, where the shared library is placed and what type the function is. Additionally, I add <code>&#36;KernelID</code> and <code>&#36;ProcessID</code>:</p> <pre><code>printCFuncLibrary[HoldPattern[CompiledFunction[__, lib_]]] := StringJoin["{KernelID: ", ToString[$KernelID], ", ProcessID: ", ToString[$ProcessID], "} -&gt; ", ToString[lib, InputForm]] </code></pre> <p>Using this on <code>f</code> and I get</p> <pre><code>printCFuncLibrary[f] (* {KernelID: 0, ProcessID: 3809} -&gt; LibraryFunction["/home/patrick/.Mathematica/ApplicationData/\ CCompilerDriver/BuildFolder/lenerd-3809/compiledFunction0.so", "compiledFunction0", {{Real, 0, "Constant"}}, Real] *) </code></pre> <p>Please note that that the build-folder has the process id of my main-kernel in it: "lenerd-3809". If you try now to execute this on different sub-kernels, you see that shared library function which is used stays the same. This is kind of expected:</p> <pre><code>ParallelTry[printCFuncLibrary, {f, f, f, f}, 4] </code></pre> <p>What <em>is</em> kind of unexpected is, that when I call <code>f</code> even on only 1 sub-kernel, I lose the vector-parallelization completely and only one processor is working on the task</p> <pre><code>ParallelTry[f, {data}, 1]; </code></pre> <p><img src="https://i.stack.imgur.com/hnX96.png" alt="enter image description here"></p> <p>I would have expected, that when calling the compiled function (compiled on the main-kernel) in sub-kernels, that there are some clashes whatsoever.</p> <h2>Compiling the function <strong>on</strong> the sub-kernels</h2> <p>Since I could not explain the above behavior, but I could surely imagine, that having only one shared library function which is maybe only loaded one time, is not the best situation when several processes want to access it.</p> <p>But why not compile the function on all sub-kernels. With this every sub-kernel gets its own copy of the shared library and loads its own version of the function:</p> <pre><code>ParallelEvaluate[ fsub = Compile[{{t, _Real, 0}}, #, CompilationTarget -&gt; "C", RuntimeAttributes -&gt; {Listable}, Parallelization -&gt; True] &amp;@ Sum[Sin[2.0 Pi k t]/k, {k, 1000}]; ]; ParallelEvaluate[printCFuncLibrary[fsub]] </code></pre> <p>I skip my output here, but what you should see is, that every sub-kernel gets its own copy of the shared library, place in a folder which is named like the process id of the sub-kernel. Additionally, on our main-kernel there doesn't exist a function <code>fsub</code> and therefore calling it with a numeric value stays unevaluated. On the other hand, <code>ParallelEvaluate[fsub[.1]]</code> calculates the correct results.</p> <p>If you now try to supply the vector <code>data</code> to the compiled function on the sub-kernel you see, that this is not processed parallel</p> <pre><code>ParallelTry[fsub, {data}]; </code></pre> <p>I tried several other things to get some insight in the behavior, but nothing really helped me to understand, what's going on.</p> <h2>You might ask...</h2> <p>... when your compiled function is parallelized so nicely, isn't it pretty useless, to take a second layer of parallelization? The answer is, yes, but for my real problem this is not the case. Assume you have a minimization problem and you parallelize your target-function only. Still, since the minimization method runs serially and only the calls to the target-function are parallel, there is still much cpu-time left. In such a cases, it would be reasonable to run two or more minimizations at the same time.</p> https://mathematica.stackexchange.com/q/5470 16 Speeding up random walk for many particles Paul https://mathematica.stackexchange.com/users/1233 2012-05-11T22:28:19Z 2012-05-12T00:50:28Z <p>I am trying to speed up this code for many particles to take a random walk. I'm not sure why it is so slow for such a simple task. </p> <p>I got a few hints from colleagues to reduce the precision of the calculations, compile some of the functions, parallelize the code, or implement C code. I tried to implement each of these, but I most likely don't know what I'm doing because I couldn't get it to work. The particles can and should be simulated in parallel to take advantage of the two cores.</p> <p>I have a bunch (10,000) particles that each take 1000 fixed steps in random directions. I have a bunch of lists with the length of number of particles, and I operate on those lists. This is faster than doing a nested loop, where I first loop over every particle and then loop over each time step. Two random angles are generated for each particle at each time step. Each particle's position is then incremented. I also want to know how many times each particle has crossed a spherical shell. The history of each particle is important: eventually, I will distinguish particles by the number of times they cross the shell so that they take bigger steps the more they have crossed.</p> <p>I tried timing each step, and it seems like the trig functions slow things down. Is there any way to speed this up? Is there a better way to write this code to make it much faster? Eventually, I would like to add some more complications to this code and run it for many more time steps and particles. So, any general speed tips would help.</p> <p>For comparison, the absolute timing of the Do loop is 3 seconds on my laptop with Core i5 M 540 @ 2.53 GHz, with 4 GB RAM. </p> <pre><code>numparticles = 10^4; numsteps = 10^3; radius = 1.;(*radius of shell*) particlesx = Table[1.001, {numparticles}];(*x-coordinate for each particle*) particlesy = Table[0., {numparticles}];(*y-coordinate for each particle*) particlesz = Table[0., {numparticles}]; numcrossings = Table[0., {numparticles}];(*counts the number of crossings for each particle*) Do[ \[Theta]rand = RandomReal[{0., Pi}, numparticles];(*random polar angle for each particle*) \[Phi]rand = RandomReal[{0, 2.*Pi}, numparticles];(*random azimuthal angle for each particle*) rold = particlesx^2 + particlesy^2 + particlesz^2;(*original distance from origin for each particle*) particlesx = particlesx + 0.01*Sin[\[Theta]rand]*Cos[\[Phi]rand];(*update each particle's position*) particlesy = particlesy + 0.01*Sin[\[Theta]rand]*Sin[\[Phi]rand]; particlesz = particlesz + 0.01*Cos[\[Theta]rand]; rnew = particlesx^2 + particlesy^2 + particlesz^2;(*new distance from origin*) isitoutside = (rold - radius)*(rnew - radius);(*yields negative number if the particle crossed the sphere, else positive*) switchsides = Map[If[# &lt; 0, 1, 0] &amp;, isitoutside];(*if the particle crossed, it increments numcrossings*) numcrossings = numcrossings + switchsides; , {step, 1, numsteps}]; // AbsoluteTiming </code></pre> https://mathematica.stackexchange.com/q/11595 16 Package found with Needs, but not with ParallelNeeds celtschk https://mathematica.stackexchange.com/users/129 2012-10-05T10:27:39Z 2016-11-27T21:21:19Z <p>I want to use a self-written package in a parallel computation. However while <code>Needs["mypackage"]</code> works without problems, <code>ParallelNeeds["mypackage`"]</code> can't open my package.</p> <p>The immediate reason is related to the fact that I added to <code>$Path</code>, and the package is in one of the directories I've added. <code>ParallelTable[$Path,{1}]</code> reveals that this addition is not propagated to the subkernels, despite the fact that it is changed in the user kernel init file (<code>$UserBaseDirectory/Kernel/init.m</code>), and thus long before any subkernels are launched in the Notebook. Explicitly using <code>DistributeDefinitions[$Path]</code> didn't help either.</p> <p>So my question is: Why isn't the \$Path propagated to the kernels, and more importantly, what can I do about it?</p> https://mathematica.stackexchange.com/q/1620 16 Parallelizing Numerical Integration in Mathematica rprospero https://mathematica.stackexchange.com/users/501 2012-02-10T23:06:05Z 2012-02-13T17:49:30Z <p>I have an ugly, six dimensional function that I need to integrate numerically. It works, but it currently take twelve hours to complete the calculation. Is there any good way to parallelize the calculation to run across multiple cores?</p> <p>The best I've managed to come up with is:</p> <pre><code>Sum[ ParallelTable[ NIntegrate[calc[a,b,c,d,e,f], {a,-1,1}, {b,-1,1}, {c,-1,1}, {d,-1,1}, {e,-1,1}, {f,-1+i/4,-1+(i+1)/4}], {i,0,7}]] </code></pre> <p><strong>EDIT:</strong> The value for calc is given below:</p> <pre><code>calc[a,b,c,d,e,f] = (1.97531*10^15 (3. Q Cos[(3 Q)/20000] - 20000. Sin[(3 Q)/20000])^2)/(1.58025*10^24 + 6.32099*10^16 Q^2 + Q^6 + (-1.58025*10^24 + 7.90123*10^15 Q^2 - 3.55556*10^8 Q^4) Cos[( 3 Q)/10000] + (-4.74074*10^20 Q - 2.96296*10^12 Q^3) Sin[(3 Q)/ 10000])* UnitStep[5/2 - a]^2 UnitStep[5/2 + a]^2 UnitStep[5 - b]^2 UnitStep[ 5 + b]^2 UnitStep[5 - a - 1944 c] UnitStep[5 + a + 1944 c] UnitStep[ 5 - b - 1944 d] UnitStep[5 + b + 1944 d] UnitStep[ 40 + a - 1600 e] UnitStep[40 - a + 1600 e] UnitStep[ 45/2 + b - 1600 f] UnitStep[45/2 - b + 1600 f] /. Q-&gt; (8000000 \[Pi] Sin[ 1/2 ArcCos[ 1/2 Sqrt[Cos[2 c] + Cos[2 d]] Sqrt[Cos[2 e] + Cos[2 f]] + Sin[c] Sin[e] + Sin[d] Sin[f]]]) </code></pre> <p>There are two immediate things to note. First, there's a removable singularity at (c==e &amp;&amp; d==f). I've also tried using a piecewise function to plug this discontinuity, but it doesn't seem to have a significant effect on the speed.</p> <p>One other thought that has come to mind is using the UnitSteps to directly find the range of integration. I hadn't done that before mostly out of laziness and the understanding that Mathematica automatically breaks up piecewise functions for integration, anyway.</p> https://mathematica.stackexchange.com/q/1665 15 Efficient conditional Mean[] on a large data set 500 https://mathematica.stackexchange.com/users/172 2012-02-12T13:51:46Z 2012-02-13T08:37:49Z <p>Consider a list, <code>AK6</code>, that has 382 sublists of length varying from 2500 to 3000. Each "subsublist" is as such : <code>AK6[[1,1]]={5.5,1001}</code></p> <p>With <code>AK6[[All,1]]</code> going from 1001 to 4001 with gaps (missing points)</p> <p>I now need to compute the <code>Mean[]</code> for all the sublists with equal second value. That is <code>Select</code> with <code>Flatten[AK6,1]</code> all the list with 2005 as the second value (from 350 to 384 items approx)</p> <p>I do this using :</p> <pre><code>ParallelTable[ Mean[Select[Flatten[AK6s, 1], #[] == gazeNo &amp;][[All,1]]], {gazeNo, Range[1000, 4000, 1]}] </code></pre> <p>It does what I need but this is very slow. Is there a way to do this computation faster?</p> <hr> <p>Download the 13 MB list as</p> <pre><code>AK6 = Uncompress@Import["http://api2.ge.tt/0/9F8d6WD/0/blob/download", "String"]; </code></pre> https://mathematica.stackexchange.com/q/473 15 How can we implement "Sleep Sort"? Mr.Wizard https://mathematica.stackexchange.com/users/121 2012-01-22T10:46:18Z 2012-01-29T08:58:37Z <p>Inspired by <a href="https://codegolf.stackexchange.com/q/2722/826">Implement Sleep Sort</a>:</p> <blockquote> <p>Sleep Sort is an integer sorting algorithm I found on the Internet. It opens an output stream, and for each input numbers in parallel, delay for the number seconds and output that number. Because of the delays, the highest number will be outputted last.</p> </blockquote> <p>I would like to know if it is possible to implement this in <em>Mathematica</em>. As a version 7 user I would of course like to see a solution that works there, but I think that <code>RunScheduledTask</code> may be needed to accomplish this.</p>