I have a question regarding the use of ParallelSum. I would like to perform a sum of the form:

    A[[a,b,c,d,e,f]] B[[a,g,h,d,i,j]] F[[b,g,k,e,i,l]] G[[c,h,k,f,j,l]],

where A, B, F and G are 6-rank tensors with symbolic entries, i.e. their entries are not numbers, and n=10. Many of the components of these 6-rank tensors are zero. I have access to a supercomputer where I can use up to 512 processors. So, my question is the following, is there a smart way of performing this sum in Mathematica? In particular, do I understand correctly that ParallelSum only parallelises the first index just like ParallelTable? If so, can we work around this?

  • $\begingroup$ Please put the code in a code block for readability. If the symbolic sum has so many terms that you are looking at parallel computation, what is the point of getting the result anyway? What are you going to do with it? $\endgroup$ – Szabolcs Sep 21 '17 at 8:43
  • $\begingroup$ There will be a remarkable cancellation between all of these terms at the end of the day. In general, I would like to know how to make ParallelSum parallelise in more than one argument. Is that possible at all? $\endgroup$ – user12588 Sep 21 '17 at 9:02
  • $\begingroup$ The only reason why paralellizing over the first index is not sufficient is that it takes only 10 values, and you are running more than 10 subkernels. You can pre-generate index tuples for the first few variables, and loop over that list. E.g. using the first 4 would give you 10^3=1000 computations to distribute to different subkernels. That said, I am extremely skeptical about your application, and doubt that it will work any better than a serial evaluation would. $\endgroup$ – Szabolcs Sep 21 '17 at 9:14
  • $\begingroup$ I share @Szabolcs' doubts. You might be better off by replacing your sums by suitable combinations of Transpose and Dot or by TensorContract. $\endgroup$ – Henrik Schumacher Sep 21 '17 at 9:16

This is strictly to answer the question:

In general, I would like to know how to make ParallelSum parallelise in more than one argument. Is that possible at all?

Generally, it is sufficient to paralellize only over the first index. Often it is even preferable, as breaking the computation into too many parts can increase the communication overhead between the main kernels and the subkernels.

In your case, the first variable takes only 10 values. This means that such a calculation cannot make use of more than 10 subkernels. As a workaround you can "flatten" the first few variables by pre-computing index-tuples for them.

Something like this (untested):

allTuples = Tuples[Range[n], 3];

 Module[{a, b, c},
  {a, b, c} = tuple;
  Sum[ (* non-parallel *)
   A[[a, b, c, d, e, f]] B[[a, g, h, d, i, j]] F[[b, g, k, e, i, l]] G[[c, h, k, f, j, l]],
   (* a,b,c moved to the outer loop *) {d, 1, n}, {e, 1, n}, {f, 1, n}, {g, 1, n}, {h, 1, n}, {i, 1, n}, {j, 1, n}, {k, 1, n}, {l, 1, n}
 {tuple, allTuples}

That said, I am very skeptical about this application. This calculation is likely to generate huge symbolic expressions which are useless as an end result.

Furthermore, transferring large symbolic expressions between the main kernel and the subkernels is quite slow. That means that the parallelization overhead will be very large (unless the cancellation you mention is truly amazing and each partial result is tiny).

  • $\begingroup$ Thanks for the answer Szabolcs. I am inclined to agree with the parallelisation overhead. Do you know if Sum or ParallelSum could be made faster if these 6-rank tensors are declared as SparseArrays? As I said, many of the components of these tensors are zero. $\endgroup$ – user12588 Sep 21 '17 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.