6
$\begingroup$

For some reason, I'd like to use ParallelTable with a variable number of iterators.

  • Table[a[1], {a[1], 0, 10}] works fine:

Output: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

  • ParallelTable[a[1], {a[1], 0, 10}] returns an error:

Output: ParallelTable::nopar1: ParallelTable[ci[1],{ci[1],0,10}] cannot be parallelized; proceeding with sequential evaluation. >>

{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

I am also interested in workarounds to use ParallelTable and a variable number of indices (typically: ParallelTable[f[Table[a[i],{i,1,n}]],Table[a[j],{j,1,n}]] where f is a function of a list with variable size).

$\endgroup$
6
  • $\begingroup$ It appears ParallelTable[a[1], {a[1], 0, 10}] not working with variable like a[1], since ParallelTable[f[i], {i, 0, 10}] does work. `ParallelTable[f[Table[a[i],{i,1,n}]],Table[a[j],{j,1,n}]] does not make sense. $\endgroup$
    – Kattern
    Jun 6, 2015 at 4:54
  • $\begingroup$ @kattern: Yes, that's my question: why does it work with Table but not with ParallelTable. With[{iter = Sequence @@ Table[{a[j],-1, n}, {j, 1, n}]}, Table[f[Table[a[i], {i, 1, n}]], iter]] if you prefer... $\endgroup$
    – anderstood
    Jun 6, 2015 at 4:56
  • $\begingroup$ Still do not understand, can you please write an executable Table version of what you what? $\endgroup$
    – Kattern
    Jun 6, 2015 at 5:01
  • $\begingroup$ @kattern Try this standalone code (which works): With[{iter = Sequence @@ Table[{gag[j], -1, 1}, {j, 1, 3}]}, Table[f[Table[gag[i], {i, 1, 3}]], iter]]!. How can I make it work with ParallelTableinstead of Table? $\endgroup$
    – anderstood
    Jun 6, 2015 at 5:02
  • $\begingroup$ @kattern: Well... changing Table to ParallelTable works in this case. I still don't get it, why does ParallelTable[a[1], {a[1], 0, 10}]not work? $\endgroup$
    – anderstood
    Jun 6, 2015 at 5:05

2 Answers 2

6
$\begingroup$

Using Trace we can see that the evaluation of ParallelTable[a[1], {a[1], 0, 10}] becomes:

Parallel`Combine`Private`parallelIterateE[
  ParallelTable, Table, Join, Identity, a[1], {a[1], 0, 10}, {Automatic, "Global`"}
]

Further using PrintDefinitions in the GeneralUtilities package lets us peek behind the curtain:

Needs["GeneralUtilities`"]

PrintDefinitions @ Parallel`Combine`Private`parallelIterateE

We see definitions specifically wanting a Symbol:

Parallel`Combine`Private`parallelIterateE[orig_, iter_, comb_, f_, 
  expr_, {i_Symbol, vals_List}, others___, {meth_, dist_, ___}] := . . .

Parallel`Combine`Private`parallelIterateE[orig_, iter_, comb_, f_, expr_, 
  it : {i_Symbol, w0_ : 1, w1_, dw_ : 1}, others___, {meth_, dist_, ___}] := . . .

If in each of these we replace i_Symbol with i_Symbol | i : _Symbol[___] and evaluate the cells we find that the problem is resolved:

ParallelTable[a[1], {a[1], 0, 10}]    (* no error message *)
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

In light of this I think it was an accident to restrict the iterators to bare Symbols, and I propose tagging this as a bug.

$\endgroup$
6
  • $\begingroup$ This is definitely an unwanted feature, as generating arbitrary-dimensional iterators and other such variables is often done with constructs like c /@ Range[dim], which is much less cumbersome than working with lists of Symbol[...], Unique[], and such. $\endgroup$
    – kirma
    Jun 24, 2015 at 13:34
  • $\begingroup$ Very enlightening, thank you. I don't know how to report a bug. $\endgroup$
    – anderstood
    Jun 24, 2015 at 14:44
  • $\begingroup$ @anderstood You're welcome. I think under the Help menu you should find an item Give Feedback... that can be used to report bugs, among other things. Or try emailing [email protected]. $\endgroup$
    – Mr.Wizard
    Jun 25, 2015 at 5:30
  • 1
    $\begingroup$ I sent them a message with Give Feedback. They answered today: I have filed a report with our developers on this issue with ParallelTable. This may allow them to address it in a future version of Mathematica. $\endgroup$
    – anderstood
    Jun 26, 2015 at 14:20
  • 1
    $\begingroup$ @anderstood Have you heard back from them on this one? The original code works just fine on v10.4. $\endgroup$ Mar 23, 2016 at 14:42
5
$\begingroup$

This seems to just be a limitation of ParallelTable, can't comment whether that has a deeper reason due to parallelism or is just a simple oversight. I think it was not possible to use expressions like a[i] as e.g. iterators in older versions but in newer version that has been added as a feature to many functions, but obviously not ParallelTable (as of 10.0.2, still doesn't work in 10.1.0). That is somewhat inconsistent, and it might work with future versions, but for the moment you'd have to use a workaround as Kattern has suggested and use explicit symbols instead of the a[i] like e.g.:

asym[i_Integer] := Symbol["a" <> ToString[i]];
With[{
    iter = Sequence @@ Table[{asym[j], -1, 1}, {j, 1, 3}]
  },
  ParallelTable[$KernelID -> f[Table[asym[i], {i, 1, 3}]], iter]
]

alternatives to create symbol names would include:

asym[i_Integer] := asym[i] = Module[{a}, a];
asym[i_Integer] := asym[i] = Unique["a"];

each has there advantages and disadvantages, a common problem is that you have to manually cleanup as they create a bunch of symbols instead of just one as your original approach...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.