Questions tagged [parallelization]
Questions on parallel computing in Mathematica.
847
questions
2
votes
1
answer
81
views
Different output when using ParallelDo
I think this is a very naive question which emerges from my poor understanding of ParallelDo and parallel computation in general.
Suppose, I have this simple code:
<...
- 1,050
0
votes
1
answer
90
views
Problem with file locking
Before reporting this to Wolfram I want to make sure that I am not mistaken. I am using MMA version 13.0.1
The following will lock a file and then write "Hello" into it, all in the current ...
- 24.8k
0
votes
0
answers
51
views
Syntax for parallel processing
I have a Do[] loop that references a few global variables but doesn't modify anything. The only output is appending data to an external file. The order of appending doesn't matter, I just can't have ...
0
votes
1
answer
44
views
Parallelization of indepedent Do
I'm struggling understanding the syntax for parallel process. I have two Do that are independent and I would like to parallelize them,
...
0
votes
0
answers
32
views
Why is the parallelized version so much slower here?
I evaluated the following code: spikes = Select[waveforms[[channel]], #[[2]] < thresholds[[channel]] &]
and according to ...
- 285
4
votes
1
answer
73
views
Problem with labeled functions and ParallelEvaluate [duplicate]
Bug introduced in 8 or earlier and persisting through 13.0.1
I came across behaviour of ParallelEvaluate that I do not understand.
In a loop I define five ...
- 903
0
votes
1
answer
74
views
Parallelize for loop in mathematica
I have the following code which essentially consists of a first for loop that iterates through a very large number of rows and a small for loop that iterates through a small number of columns. The ...
- 111
2
votes
1
answer
119
views
Speed Up the Gelman-Rubin-History function
A bit of background information
I have created a function to calculate the convergence of the Gelman Rubin diagnostic (useful in Bayesian analysis). The function seems to work as I want, however I ...
- 346
0
votes
0
answers
17
views
Context problems with ParallelTable and Import/Export
I work with notebooks with contexts unique to each notebook and I want to compute a list of functions $f(z,w)$ via ParallelTable and save them to files in NotebookA ...
- 1,114
3
votes
0
answers
57
views
5
votes
3
answers
246
views
ParallelDo barely faster than Do
I am new to parallel computing, so I am trying to get a grasp as to how this works. Many posts seem to indicate that overhead (like here) is a common pitfall, but I do not see why this would be the ...
- 1,043
1
vote
0
answers
44
views
4
votes
2
answers
84
views
Limit the number of cores used
Sometimes, when Mathematica thinks it is needed, it automatically threads a computation on several cores, it can happen for NIntegrate or ...
- 6,124
0
votes
1
answer
80
views
Vectorise IF statement
I would like to perform an iteration on a list, component-wise. The iteration is time consuming and it would be hence beneficial to be able to stop it once an element of the list has reached a ...
- 737
2
votes
0
answers
84
views
How to avoid running out of memory in ParallelMap?
I am mapping a function over a list of length 77237.
ParallelMap[AnalyzeZernikeRecord,records]
The result of each function application is just a list of 12 numbers....
- 1,857
1
vote
0
answers
64
views
How can I make ParallelDo faster than Do?
I am trying to optimize my code as much as possible. So first I used a compile and now I am trying to parallelize the code in order to make it faster because in the end I will need to make a loop that ...
- 21
4
votes
1
answer
190
views
Speeding up Fourier transform and eigenvector calculation
I have a matrix $M$ in Bloch basis (momentum space). Then I have to take Fourier transform of $M$ to go to real space. The size of this lattice in real space can be fixed to some value $N_{unit}$.
...
- 601
1
vote
2
answers
79
views
Parallelize elements of a vector or array
Suppose I want to calculate a vector or an array (say, for simplicity a 4-by-1 vector), but each element contains independent numerically evaluated functions $f_1,f_2,f_3,f_4$ and each takes long time....
- 240
1
vote
0
answers
48
views
Possible bug: Wrong differentiation of ParametricFunction inside Parallelize?
In the note RootSearch problem when called from within ParallelDo
I described the problem with Ted Ersek's RootSearch package and came to the conclusion that it is ...
- 535
1
vote
0
answers
108
views
RootSearch problem when called from within ParallelDo
I have been using Ted Ersek's RootSearch package for Wolfram Mathematica for many years. The package worked great, including inside the ...
- 535
4
votes
1
answer
120
views
ParallelTable/ParallelDo for Newton's Method doesn't work [closed]
We run such ParallelTable based on @Syed https://mathematica.stackexchange.com/a/261603/54516.
...
- 1,746
0
votes
1
answer
86
views
Trying to speed up finding solutions of quadratic form related to the weight of a matrix
Say I have an integer matrix of size $5 \times 5$ - I am trying to find the solutions to the equation $n^4 wtM = n w^2 + x1^2 + x2^2 +x3^2 +x4^2 + x5^2$ subject to the constraints $x1+x2+x3+x4+x5 = 0$ ...
- 495
2
votes
0
answers
64
views
Set Default System Options
I have a new Mac M1 machine and am getting Mathematica set up on it. I noticed that when I run some simple benchmarking tests, for example matrix diagonalization, the code runs significantly slower if ...
- 21
7
votes
0
answers
153
views
Are all MMA computations single-core unless explicitly parallelized?
I was wondering if the WolframMark Benchmark uses more than one core. This becomes relevant when comparing Benchmark scores between, say, 4-core and 8-core machines.
I looked at Benchmark's code, and ...
- 3,456
3
votes
1
answer
76
views
Parallel Cellprint Not Rendered
I'd like to parallelly construct cells (possibly dynamically updating ones) and output render them on the front-end. However, I found this is not working.
For example,
...
- 777
4
votes
0
answers
61
views
A WolframKernel (with PID) is created by a LocalSubmit. After the LocalSubmit finishes its job, the kernel remains. How to kill it within Mathematica?
When defining a task with LocalSubmit, a new subkernel is created. When you remove the task after the task is done, the task is removed but the subkernel remains ...
- 375
0
votes
1
answer
70
views
Parallelization in While[] loop when testing if a condition is met
Well, I am trying to run the following code:
...
- 1,723
0
votes
0
answers
76
views
Effectively using all CPU cores while exporting a sequence of plots
In Wolfram Mathematica 12.3.1 on macOS I use
Do[Export i-th plot to PNG file,{i,LargeNumber}]
to generate a sequence of plots. This works well, but I noticed that ...
- 339
0
votes
0
answers
35
views
Parallel Table dont use Paralization after long calculation
at first: This is my first post and I hope you understand me. If I made any mistakes how to write a question please feel free to correct me. Now to the question.
I wrote a module which evaluates a ...
- 1
2
votes
0
answers
41
views
What is the difference between "SessionSubmit" and "LocalSubmit" in performance?
Does SessionSubmit forward the computation to the master thread and LocalSubmit to some other idle thread? By thread, I mean CPU ...
- 375
9
votes
2
answers
356
views
How to parallelize and speedup AppendTo? [duplicate]
Say I have a list of original data (for now generating random list as follows):
...
- 157
0
votes
0
answers
45
views
Launch remote kernels on the cluster node
I run the Mathematica front end with Xterm on main machine of Unix cluster. It can start parallel kernels locally
...
- 783
2
votes
0
answers
51
views
Parallel kernel configuration settings missing on MacBook ARM
$Version
(* 12.3.1 for Mac OS X ARM (64-bit) (July 8, 2021) *)
In the preferences panel, the parallel kernel settings are missing.
Evaluating
...
- 9,318
1
vote
1
answer
57
views
Data parallelism for independent probabilistic trials
I'm considering the best way to implement the data parallelism features of Mathematica for a specific task. To that end I have a minimal code example below which contains the basic features of my ...
- 271
1
vote
0
answers
90
views
Increase the speed of Machine learning based on 'Predict'
I am training a ANN model with a large dataset (~million level) using Predict or NetTrain function. I do not have a GPU machine but have a server equipped with 16 CPU cores. How can I maximize the ...
0
votes
1
answer
41
views
Cannot retrieve a function definition from file after save with ParallelDo
This is an example of a problem I have retrieving functions from a file I save using ParalleDo.
...
- 1,114
4
votes
1
answer
68
views
Why are DownValues of symbols in System` such as CircleDot not distributed to subkernels?
Consider a symbol in System such as CircleDot. I want to redefine this symbol and distribute the definition to subkernel, to use ...
- 7,299
5
votes
0
answers
191
views
How to allow one process to take advantage of all available cores while other processes run in parallel?
I have to run multiple tasks simultaneously:
Data download from stock exchange during live market session and append it to a global variable.
Process the data by taking the data from the front of the ...
- 91
0
votes
0
answers
60
views
MathLink related error on HPC
Most of the times, Mathematica gives the following error for the jobs submitted on multiple nodes in queue:
...
- 902
1
vote
0
answers
58
views
What do do if Mathematica stops without error messages?
I am trying to run the relatively simple code
...
1
vote
0
answers
55
views
Is FindMinimum function multithreaded? Is it worth it to use this function in ParallelDo loop? [duplicate]
I have to call FindMinimum function n times on my data.
But when I put it in ParallelDo loop it works slower than in casual Do loop.
I wonder what might be the case (in the loop there are no other ...
1
vote
0
answers
145
views
How to limit the cpu usage of Mathematica?
I am runing mathematica with ParallelMap[], on a win10 computer with 8 physical cores.
The cpu usage always rocks up to 100%, though ...
- 57
0
votes
1
answer
30
views
How to create ParallelDo loop with n dimensions of evaluation?
For example if I have n=2 i get
ParallelDo[something, {v[1], c[1]}, {v[2], c[2]}]
and if I have n=3 I get
...
3
votes
1
answer
306
views
Why ParallelTable is even slower than Table for large tensor construction?
I would like to create a large rank-4 tensor by using both Table and ParallelTable.
What is actually constructed is given below:
...
- 425
2
votes
0
answers
121
views
Want to revert back from version 12.3 to 12.1 but kernels no longer run in 12.1 [closed]
I upgraded from 12.1 to 12.3 and then Mathematica began randomly crashing intermittently. It's not reproducible: it crashed when I was rotating a 3D plot twice, crashed once when I tried to save a ...
- 21
4
votes
1
answer
122
views
How can I reduce the memory cost of this parallel table?
The main question
I was trying to obtain the piecewise cubic-spline interpolating polynomials of a series of very long lists of (x, f(x)) pairs (O(100) of them with length ~350,000), i.e. to get ...
- 41
1
vote
0
answers
33
views
Running ParallelTable[] with ssh
I have a ParallelTable[] code, that runs perfectly fine on my local machine, but to speed things up a notch, I want to run it on a remote server. I configured everything in Evaluation>> Parallel ...
- 341
2
votes
1
answer
84
views
Considerations for Compiling and Parallelizing Code Outside or Inside Map[]
Assume f as well as Map[f,list] are parallelizable and compilable, such that parallelization and compilation give significant ...
- 1,577
0
votes
0
answers
57
views
parallelization works only with one kernel
I am using MMA 12.3.1 on Ubuntu 18.04, and my license support parallelization. The default kernel is Local and I create another one named newKernel. when I launch MMA and choose Local to perform some ...
- 3,102
1
vote
0
answers
38
views
Should I use `ParallelNeeds` or `DistributedContexts`?
According to Mathematica documentation, ParallelNeeds can load packages for all kernels, while parallelized functions like ...
- 489