I would like to build up a 2D-Table using ParallelTable as follows

tab = Flatten[ParallelTable[{i,j,f[i,j]},{i,1,ni},{j,1,nj}],1];

This instruction works perfectly. However, I would like to speed it up. Indeed, the function f[i,j] is defined as f[i_,j_] := f1[i] f2[i,j]

The particularity here is that f1[i] is expensive to compute, so that I would like to compute it only one time for each i.

How would it be possible to tune the ParallelTable so that f1[i] is only evaluated when it has never been done before ? A kind of nested ParallelTable ? Using memoization ? (As the construction of the table remains long, it is key for me to necessarily use parallelized evaluations on multiples cores)

  • 2
    $\begingroup$ I "think", don't have time to try right now, f[i_]:=f[i]=body of f to memoize and SetSharedFunction[f] will do what you want. The down values of f[i] should be shared. Not really sure who wins if two kernels try to set f[1] at the same time. I assume first wins and both would calculate the same value unless you're doing some monte-carlo? $\endgroup$ – Ymareth May 12 '14 at 7:57
  • $\begingroup$ Indeed, my functions are completely deterministic, so that f[1] has only value and I don't care which kernel calculates it and when. I will try your approach. Otherwise, I thought I might proceed manually and define a function listf[i_] := v = f1[i]; Table[{v f2[i,j]},{j,1,nj}] and then use ParallelTable[{i,j,listf[i]},{i,1,ni}] (The list structure is not good, but this is the idea.) However, it might not be as fast as expected. $\endgroup$ – jibe May 12 '14 at 8:13
  • $\begingroup$ @Ymareth Your solution worked ! I defined the expensive function f1 using the recurrence syntax f1[i_] := f1[i] = ... and added the instruction SetSharedFunction[f1]. Using the instruction Print[$KernelID], I also checked that the evaluations were correctly parallelized. I divided by a factor of 50 my computation time... Should I/you write an answer to my question, or are these comments enough ? $\endgroup$ – jibe May 12 '14 at 9:03
  • $\begingroup$ I'll try to write a proper answer in next few days, unless someone beats me to it - thanks :). $\endgroup$ – Ymareth May 12 '14 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.