2
$\begingroup$
apple = {1, 2, 3};
sapple = 1;
banana = {10, 20, 30};
sbanana = 10;
kiwi = {100, 200, 300};
skiwi = 100;
data = {"apple", "banana", "kiwi"};

myfun[data_, scale_] := Total[data]/scale
myfun[apple, sapple]
myfun[banana, sbanana]
myfun[kiwi, skiwi]
myfun[ToExpression[#], ToExpression["s" <> ToString[#]]] & /@ data

In parallel, it still works, but with a warning:

ParallelMap[myfun[ToExpression[#], ToExpression["s" <> ToString[#]]] &, data]

enter image description here

What's the correct way of coding this?

WARNING:

@AlbertRetey commented below that

it probably is worth mentioning that while you do get the expected result the code is not evaluated in the parallel kernels but on the master, which is most probably not what you intended. What happens is that the parallel kernels return the unevaluated expressions which then are evaluated on the master...

I am not sure whether this is true. However, I think this is an extremely important observation as MMA DOES NOT tell you this. So users will be cheated by the impression that MMA still works in parallel.

$\endgroup$
  • 1
    $\begingroup$ I suspect you need to DistributeDefintions for apple, sapple... etc. They are only defined on your master kernel, not the slaves. $\endgroup$ – Ymareth Jan 14 '15 at 15:02
  • $\begingroup$ Related: (6511) (See Related links there too.) $\endgroup$ – Mr.Wizard Jan 14 '15 at 15:11
  • 1
    $\begingroup$ it probably is worth mentioning that while you do get the expected result the code is not evaluated in the parallel kernels but on the master, which is most probably not what you intended. What happens is that the parallel kernels return the unevaluated expressions which then are evaluated on the master... $\endgroup$ – Albert Retey Jan 16 '15 at 9:44
  • 1
    $\begingroup$ @Mr.Wizard: I'm just talking about the OPs troubled code. Your code seems to be OK. I first was tricked to believe it had the same problem but it doesn't: the reason is that in your case ParallelMap sees the symbols d and s and can autodistribute them to the parallel kernels... $\endgroup$ – Albert Retey Jan 17 '15 at 1:47
  • 2
    $\begingroup$ @Albert Okay, I'm glad we're on the same page. Yes, that's how my code fixed the OP's problem, even though I didn't state it. Perhaps I should have. Nevertheless there are other reason to prefer "indexed objects" (DownValues) over a long list of Symbols, so I chose to simply recommend the (IMO) superior format without justification. As always I would be happy to attempt to explain further if asked. $\endgroup$ – Mr.Wizard Jan 17 '15 at 1:50
3
$\begingroup$

I would rethink your data format. Consider using "indexed objects" (DownValues) or perhaps Associations. One example:

d["apple"]  = {1, 2, 3};
s["apple"]  = 1;
d["banana"] = {10, 20, 30};
s["banana"] = 10;
d["kiwi"]   = {100, 200, 300};
s["kiwi"]   = 100;
data        = {"apple", "banana", "kiwi"};

myfun[data_, scale_] := Total[data]/scale

ParallelMap[myfun[d[#], s[#]] &, data]
{6, 6, 6}

Evidence that my code is running in parallel:

ParallelMap[(Pause[1]; myfun[d[#], s[#]]) &, data] // AbsoluteTiming

{1.016058, {6, 6, 6}}

Manual timing also confirms this result.

$\endgroup$
  • $\begingroup$ Thanks! I will wait to see if there is a better alternative. otherwise, I might have to rewrite hundreds of datasets using the DownValues. $\endgroup$ – Chen Stats Yu Jan 14 '15 at 15:51
  • $\begingroup$ @Chen Please don't feel rushed to Accept this answer. Take your time; you may like other answered better. I will say however that in the long run it is much easier to access data by a list of keys than a list of Symbols that constantly "want" to evaluate at the wrong time. $\endgroup$ – Mr.Wizard Jan 14 '15 at 19:08
  • 1
    $\begingroup$ I can always change my mind about the correct answer :). For now, i used a loop to loop all data into the DownValues. So it's kind of sorted out. $\endgroup$ – Chen Stats Yu Jan 14 '15 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.