4
$\begingroup$

I would like to build up a 2D-Table using ParallelTable as follows

tab = Flatten[ParallelTable[{i,j,f[i,j]},{i,1,ni},{j,1,nj}],1];

This instruction works perfectly. However, I would like to speed it up. Indeed, the function f[i,j] is defined as f[i_,j_] := f1[i] f2[i,j]

The particularity here is that f1[i] is expensive to compute, so that I would like to compute it only one time for each i.

How would it be possible to tune the ParallelTable so that f1[i] is only evaluated when it has never been done before ? A kind of nested ParallelTable ? Using memoization ? (As the construction of the table remains long, it is key for me to necessarily use parallelized evaluations on multiples cores)

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ I "think", don't have time to try right now, f[i_]:=f[i]=body of f to memoize and SetSharedFunction[f] will do what you want. The down values of f[i] should be shared. Not really sure who wins if two kernels try to set f[1] at the same time. I assume first wins and both would calculate the same value unless you're doing some monte-carlo? $\endgroup$
    – Ymareth
    May 12, 2014 at 7:57
  • $\begingroup$ Indeed, my functions are completely deterministic, so that f[1] has only value and I don't care which kernel calculates it and when. I will try your approach. Otherwise, I thought I might proceed manually and define a function listf[i_] := v = f1[i]; Table[{v f2[i,j]},{j,1,nj}] and then use ParallelTable[{i,j,listf[i]},{i,1,ni}] (The list structure is not good, but this is the idea.) However, it might not be as fast as expected. $\endgroup$
    – jibe
    May 12, 2014 at 8:13
  • 1
    $\begingroup$ @Ymareth Your solution worked ! I defined the expensive function f1 using the recurrence syntax f1[i_] := f1[i] = ... and added the instruction SetSharedFunction[f1]. Using the instruction Print[$KernelID], I also checked that the evaluations were correctly parallelized. I divided by a factor of 50 my computation time... Should I/you write an answer to my question, or are these comments enough ? $\endgroup$
    – jibe
    May 12, 2014 at 9:03
  • 1
    $\begingroup$ I'll try to write a proper answer in next few days, unless someone beats me to it - thanks :). $\endgroup$
    – Ymareth
    May 12, 2014 at 9:32

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.