0
$\begingroup$

I have three large lists that need to have certain values removed from them. Each list is completely independent of each other, so three threads should be able to execute these in parallel. Here's a MWE of what I want to do:

In serial:

maxN = 1000000;
list1=RandomReal[1,maxN];
list2=RandomReal[1,maxN];
list3=RandomReal[1,maxN];
list1=DeleteCases[list1,x_/;x>=0.5];
list2=DeleteCases[list2,x_/;x>=0.5];
list3=DeleteCases[list3,x_/;x>=0.5];

As you can see, it is very straight forward. I have tried wrapping the entire block of "DeleteCases" with Parallelize and ParallelTry but it shows no speedup. In fact, it actually slows it down a bit. Of course, I am using LaunchKernels[] first, which loads 4 kernels (I have 1 physical CPU, 4 physical cores, 8 logical cores).

Is there a way I can execute each line on a different kernel such that I can evaluate a single cell and have it fork the jobs to 3 kernels? (i.e. this is a piece of larger cell which I do not want to split between cells or other notebooks)

Again, this is a MWE. I do not have 3 simple lists like this, but I do want to delete specific cases from each list. That's the part I need to parallelize.

This is running on Mathematica 9, Win 10 x64.

Thanks!

$\endgroup$
3
  • $\begingroup$ try maxN = 1000000; list = Table[RandomReal[1, maxN], 3]; a = ParallelTable[ DeleteCases[list[[i]], x_ /; x >= 0.5], {i, 1, 3}]; does it work? $\endgroup$
    – Alucard
    Aug 10, 2017 at 17:07
  • $\begingroup$ Sort of... need to replaced 3 with {3} and then it'll run. However, I need to save the results of the output, and that doesn't happen. I will modify above to show what I need. Ok: so say I used ParallelTable[ list[[i]]=DeleteCases[list[[i]], x_ /; x >= 0.5], {i, 1, 3}] It still doesn't actually save it to list[[i]] for some reason... $\endgroup$
    – Matt Stein
    Aug 10, 2017 at 17:28
  • $\begingroup$ Ok, figured that out: create new list to store results in, set it shared, and it'll work. However, it's slower than the serial method. $\endgroup$
    – Matt Stein
    Aug 10, 2017 at 17:38

1 Answer 1

0
$\begingroup$

You may use ParallelMap if the function is the same for all lists. Otherwise you may use Parallelize with MapThread.

With

maxN = 1000000;
{list1, list2, list3} = RandomReal[1, {3, maxN}];

Same Function to All Lists

{list1, list2, list3} =
 ParallelMap[DeleteCases[#, x_ /; x >= 0.5] &, {list1, list2, list3}]

Different Function to Each List

{list1, list2, list3} =
 Parallelize[
  MapThread[#1@#2 &,
   {
    {DeleteCases[#, x_ /; x >= 0.5] &,
     DeleteCases[#, x_ /; x <= 0.4] &,
     DeleteCases[#, x_ /; x >= 0.8] &}
    ,
    {list1, list2, list3}
    }
   ]]

Hope this helps.

$\endgroup$
1
  • $\begingroup$ That works perfectly, thank you for both cases! $\endgroup$
    – Matt Stein
    Aug 11, 2017 at 1:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.