0
$\begingroup$

The following seems to work based on memory usage.

bigasc=AssociationThread[RandomInteger[10^30,10^6],Range[10^6]];
DistributeDefinitions[bigasc];
k=RandomInteger[10^30];
bigasc[k]//AbsoluteTiming

gives

out[101]= {9.37986*10^-6,Missing[KeyAbsent,754081007229273563874471056283]}

but ParallelEvaluate[bigasc[k],DistributedContexts->None]//AbsoluteTiming gives

out[102]= {1.43257,{Missing[KeyAbsent,754081007229273563874471056283],Missing[KeyAbsent,754081007229273563874471056283]}}

This seems to indicate that the entire association is passed by ParallelEvaluate even though it has attribute HoldFirst. Is there a way to properly use a distributed association so that the lookup takes place on the subkernels?

$\endgroup$
  • $\begingroup$ One should use SetDelayed instead of Set for the definitions of bigasc and k. And both bigasc and k should be distribute defined. $\endgroup$ – Wen Chern Jul 2 at 18:05
  • $\begingroup$ In my case I cannot do this. With SetDelayed the definition of bigasc would be reevaluated at every access. It takes several seconds to construct. $\endgroup$ – John McGee Jul 3 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.