1
$\begingroup$

Recently I'm trying to process some experiment data, my PC with OS Win10 (mathematica v10.4) is not enough, so I connect to another PC with OS Ubuntu 16.04 (mathematica 11.0) by the way supported here.

I hope to speed up by this way since I can use another two kernels.

However, when I use ParallelTable, I find only one local kernel works well and another three local kernels are at leisure at first, then all of the kernels seems at leisure I guess all the computation has been transferred to remote kernels, I don't know why.

At first:

At first

Later:

later

In the picture, the fist four kernels are local kernels and other two are remote kernels. Here is my code (the code used to configure remote kernel is as same as in the preceding link):

filelist = Table[FileNames["*.dat", ToString[i] <> "\\" <> ToString[j]],{i,1,3}, {j, 1, 21}];
specRow = Table[Import[filelist[[j, k, l]]], {j, 1, 3}, {k, 1, 21}, {l, 1, 401}];
od[specFileNameList_] := With[{pr = Flatten@specFileNameList[[401]]},Table[With[{spec =Flatten@specFileNameList[[k]]},Reverse[ArrayReshape[Riffle[frequency, -Log[N[(spec-bg)/(pr-bg)]]], {1024,2}]]], {k, 1, 400}]]
spec = ParallelTable[od[specRow[[i, j]]], {i, 1, 3}, {j, 1, 21}];

Here, specRow, pr and bg are lists.

Update I've change my code following Szabolcs's advice below, but situation doesn't change, here is my new code:

list = Tuples[{Range[1, 3], Range[1, 21]}];
spec = ParallelTable[Apply[od[Part[specRow, #1, #2]] &, list[[i]]], {i, 1, 63}];
$\endgroup$
  • $\begingroup$ ParallelTable[od[specRow[[i, j]]], {i, 1, 3}, {j, 1, 21}]; <- parallelization happens only according to the first iterator, {i, 1, 3}, meaning that this particular computation won't use more than 3 kernels. Workaround: Create the {i,j} pairs first, using Tuples, then ParallelMap an appropriate function over them. $\endgroup$ – Szabolcs Nov 3 '17 at 13:55
  • $\begingroup$ Thanks, I'll try it. $\endgroup$ – Bettertomo Nov 3 '17 at 13:59
  • $\begingroup$ @Szabolcs, things doesn't change if the code can call more kernels. $\endgroup$ – Bettertomo Nov 3 '17 at 15:54
1
$\begingroup$

The method supportted by remote kernel through ssh is used to compute through remote kernel only and it's the usual way we need. I misunderstood it and the situation asked in the question is the right situation. Sorry to post it in hurry.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.