ParallelMap vs building pure function on the fly

I have a case where one function builds a pure function whenever it is evaluated. A (admittedly contrived) example would be

f[l_List] := With[{funStructure = Product[l[[i]]^Slot[i], {i, Length[l]}]},funStructure & @@ l]


Using the following code works just fine

Map[f[#] &, {{2, 3}, {4, 5}}]


However, when I use ParralelMap instead of Map, I get a lot of errors:

During evaluation of In[33]:= Function::slot: Slot[i] (in ParallelKernelsPrivatesendCatch[#1,ParallelClientmakeDefinitions[LanguageDefinitionList[f->{DownValues->{<<2>>}}]]]&) should contain a non-negative integer or string.

During evaluation of In[33]:= Function::slot: Slot[i] (in ParallelKernelsPrivatesendCatch[#1,ParallelClientmakeDefinitions[LanguageDefinitionList[f->{DownValues->{<<2>>}}]]]&) should contain a non-negative integer or string.

During evaluation of In[33]:= Function::slot: Slot[i] (in ParallelKernelsPrivatesendCatch[#1,ParallelClientmakeDefinitions[LanguageDefinitionList[f->{DownValues->{<<2>>}}]]]&) should contain a non-negative integer or string.

During evaluation of In[33]:= General::stop: Further output of Function::slot will be suppressed during this calculation.

During evaluation of In[33]:= Throw::nocatch: Uncaught Throw[{ParallelProtectedneval,(ParallelKernelsPrivatesendCatch[#1,ParallelClientmakeDefinitions[LanguageDefinitionList[<<1>>->{<<1>>}]]]&)[KernelObject[1,local]]}] returned to top level.

Out[33]= Hold[
Throw[{ParallelProtectedneval, \
(ParallelKernelsPrivatesendCatch[#1,
ParallelClientmakeDefinitions[LanguageDefinitionList[\!$$\* TagBox["f", HoldForm]$$ -> {DownValues -> {HoldPattern[f[l_List]] :>
With[{funStructure = \!$$\*UnderoverscriptBox[\(\[Product]$$, $$i$$, $$Length[l]$$]
\*SuperscriptBox[$$l[[i]]$$, $$Slot[i]$$]\)}, (funStructure &) @@ l],
HoldPattern[f[x_]] :>
With[{funStructure = x #1}, (funStructure &)[
x]]}}]]] &)["KernelObject"[1, "local"]]}]]


Is there anything to be said besides "you should not be using such a function definition anyway!!!" :).

Mma Version Number is 11.1.1.0

• There seems to be a pure function being wrapped around your definition somewhere during the evaluation of ParallelMap (I don't have time to check where exactly at the moment). Since your Slot[...] expression is not wrapped inside Function, the evaluation of that pure function tries to replace the Slot[...] expression with the appropriate expression, but fails due to the index being i and not a number. You can circumvent this by "hiding" Slot[…] from Function, using e.g. Slot@@{...}. Jan 31 '19 at 21:59

Workaround

To work around the issue, simply "hide" the Slot[…] expression using Slot@@{…}:

f[l_List] := With[{funStructure = Product[l[[i]]^Slot@@{i}, {i, Length[l]}]},funStructure& @@ l]


Fix

While the above workaround is easy, a more permanent solution would be nice. The following code achieves this by fixing the underlying problem:

ParallelMap;
DownValues@ParallelProtectedkernelEvaluate =
DownValues@ParallelProtectedkernelEvaluate /. {
};


The underlying problem

As I mentioned in the comments, the issue is that the definitions are wrapped in another Function at some point. Since the Slot[…] is not inside a Function head at that point, the outer function attempts to replace the Slot[…] expression with a value (with fails of course). We can see this on the following example:

buildFunc = Hold@With[{s = Slot[1]}, s &]
(* Hold[With[{s = #1}, s &]] *)

applyFunc[f_, arg_] := ReleaseHold[f][arg]

applyFunc[buildFunc, 3]
(* 3 *)

With[{f = buildFunc}, f &@4]
(* Hold[With[{s = 4}, s &]] *)

applyFunc[With[{f = buildFunc}, f &@4], 3]
(* 4 *)


As you can see, the Slot[…] expression was replaced by 4 in the second example.

How the fix works

The fix above works by patching the offending definition of ParallelProtectedkernelEvaluate (which in this case is used to push the definitions of symbols to the parallel kernels). The patch replaces the Function with unnamed arguments with a version with named arguments. This prevents the "unprotected" Slot[…] expressions in cmd from being replaced, fixing the issue.

• Interesting, thanks for the answer. A very direct way to see the problem is Function/@Downvalues[f] Feb 1 '19 at 20:03