1
$\begingroup$

Recently I'm trying to process some experiment data, my PC with OS Win10 (mathematica v10.4) is not enough, so I connect to another PC with OS Ubuntu 16.04 (mathematica 11.0) by the way supported here.

I hope to speed up by this way since I can use another two kernels.

However, when I use ParallelTable, I find only one local kernel works well and another three local kernels are at leisure at first, then all of the kernels seems at leisure I guess all the computation has been transferred to remote kernels, I don't know why.

At first:

At first

Later:

later

In the picture, the fist four kernels are local kernels and other two are remote kernels. Here is my code (the code used to configure remote kernel is as same as in the preceding link):

filelist = Table[FileNames["*.dat", ToString[i] <> "\\" <> ToString[j]],{i,1,3}, {j, 1, 21}];
specRow = Table[Import[filelist[[j, k, l]]], {j, 1, 3}, {k, 1, 21}, {l, 1, 401}];
od[specFileNameList_] := With[{pr = Flatten@specFileNameList[[401]]},Table[With[{spec =Flatten@specFileNameList[[k]]},Reverse[ArrayReshape[Riffle[frequency, -Log[N[(spec-bg)/(pr-bg)]]], {1024,2}]]], {k, 1, 400}]]
spec = ParallelTable[od[specRow[[i, j]]], {i, 1, 3}, {j, 1, 21}];

Here, specRow, pr and bg are lists.

Update I've change my code following Szabolcs's advice below, but situation doesn't change, here is my new code:

list = Tuples[{Range[1, 3], Range[1, 21]}];
spec = ParallelTable[Apply[od[Part[specRow, #1, #2]] &, list[[i]]], {i, 1, 63}];
Improve this question
$\endgroup$
3
  • $\begingroup$ ParallelTable[od[specRow[[i, j]]], {i, 1, 3}, {j, 1, 21}]; <- parallelization happens only according to the first iterator, {i, 1, 3}, meaning that this particular computation won't use more than 3 kernels. Workaround: Create the {i,j} pairs first, using Tuples, then ParallelMap an appropriate function over them. $\endgroup$
    – Szabolcs
    Nov 3, 2017 at 13:55
  • $\begingroup$ Thanks, I'll try it. $\endgroup$
    – Bettertomo
    Nov 3, 2017 at 13:59
  • $\begingroup$ @Szabolcs, things doesn't change if the code can call more kernels. $\endgroup$
    – Bettertomo
    Nov 3, 2017 at 15:54

1 Answer 1

Reset to default
1
$\begingroup$

The method supportted by remote kernel through ssh is used to compute through remote kernel only and it's the usual way we need. I misunderstood it and the situation asked in the question is the right situation. Sorry to post it in hurry.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.