0
$\begingroup$

This question already has an answer here:

I am currently doing calculation with 7 parameters, each parameter set corresponding to a value. And I store the data in the format as a list of rule, like following

{{100, 7, 1, 1, 1, 1/5, 0} -> 0.00720852, {100, 7, 1, 2, 1, 1/5, 0} ->
   0.00456272, {100, 7, 1, 3, 1, 1/5, 0} -> 0.00363628, {100, 7, 1, 4, 1, 1/5, 0} -> 
  0.0034754, {100, 7, 1, 5, 1, 1/5, 0} ->0.00343746, {100, 7, 1, 6, 1, 1/5, 0} -> 
  0.00342957, {100, 7, 1, 7, 1, 1/5, 0} -> 0.0034},........}

I put all my calculated data into a single rule list, I name it generaldata

when I want to extract data from the generaldata used for plotting, I would write the following code for example

extractdata = Table[Table[{100, 7, line, distance, 1, 1/5, delta}, {line, 1, long + 1}, {distance, 1, 12}], {delta, 0, 1, 1/50}] /. generaldata

currently, my generaldata's length is 88000, and it took me 40 seconds to accomplish the rule replacement. It is quite slow. I also tried Parallel it

data = ParallelMap[Replace[#, generaldata] &, 
    Table[Table[{100, 7, line, distance, 1, 1/5, delta}, {line, 1, long + 1}, {distance, 1, 12}], {delta, 0, 1, 1/50}], {3}]; // AbsoluteTiming

and this took even longer about 200 seconds on an 8 core machine.

What's more, I only calculated a small part of the data I wanted. So I can foresee, if I don't improve the method, the time cost will be much more.

So I am asking here for a better quicker method.

$\endgroup$

marked as duplicate by Michael E2, Dr. belisarius, Sjoerd C. de Vries, Yves Klett, rm -rf Nov 18 '13 at 6:49

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ Duplicate: Why is this parallel evaluation with Dispatch[] so slow? In short, this is a bug in the parallel functions... using Rule in any parallel function makes it exceptionally slow. $\endgroup$ – rm -rf Nov 18 '13 at 5:05
  • $\begingroup$ @rm-rf What?! why this is a duplicate????! My main question is not why parallel is slow. My question is to get a quicker method to extract the data I wanted. $\endgroup$ – matheorem Nov 18 '13 at 8:59
  • $\begingroup$ @rm-rf maybe this is a duplicate.mathematica.stackexchange.com/questions/31613/… . I am reading now $\endgroup$ – matheorem Nov 18 '13 at 9:31
  • $\begingroup$ @rm-rf OK, I understand. Dispatch speed up rule replacement significantly. I never knew Dispatch before. Dispatch's awesome efficiency impressed me. $\endgroup$ – matheorem Nov 18 '13 at 9:43