How do you determine the optimal autocompilation length on your system

Question

When you pack lists there is an overhead therefore packing a list with, say, 2 elements is likely to cost more than you get back in efficiency. Mathematica has default list lengths for which functions creating those lists will pack the list (i.e. if the list length is less than the numbers shown below the list will not be packed):

SystemOptions["CompileOptions"]

{"CompileOptions" -> {"ApplyCompileLength" -> \[Infinity], 
   "ArrayCompileLength" -> 250, "AutoCompileAllowCoercion" -> False, 
   "AutoCompileProtectValues" -> False, "AutomaticCompile" -> False, 
   "BinaryTensorArithmetic" -> False, "CompileAllowCoercion" -> True, 
   "CompileConfirmInitializedVariables" -> True, 
   "CompiledFunctionArgumentCoercionTolerance" -> 2.10721, 
   "CompiledFunctionMaxFailures" -> 3, 
   "CompileDynamicScoping" -> False, 
   "CompileEvaluateConstants" -> True, 
   "CompileOptimizeRegisters" -> False, 
   "CompileReportCoercion" -> False, "CompileReportExternal" -> False,
    "CompileReportFailure" -> False, "CompileValuesLast" -> True, 
   "FoldCompileLength" -> 100, "InternalCompileMessages" -> False, 
   "ListableFunctionCompileLength" -> 250, "MapCompileLength" -> 100, 
   "NestCompileLength" -> 100, "NumericalAllowExternal" -> False, 
   "ProductCompileLength" -> 250, "ReuseTensorRegisters" -> True, 
   "SumCompileLength" -> 250, "SystemCompileOptimizations" -> All, 
   "TableCompileLength" -> 250}}

So, for example, if you make a list using Table

Developer`PackedArrayQ[Table[i, {i, 1, 249}]]
False

Developer`PackedArrayQ[Table[i, {i, 1, 251}]]
True

I am assuming that if you plotted the time to make uncompiled lists using Table, vs making compiled lists, the lines would intersect at ~250, beyond which packed lists become more efficient. Is that a correct interpetation of what the autocompilation length represents?

I would expect that the optimal lengths for compilation (incl. packing) vary on system to system, therefore I want to know the best way to construct a set of tests to test that proposition, and to determine the optimal list length for packing for the functions listed above.

Edit

For clarity, as per Albert's comments, there are cases when the evaluations taking place prevent compilation so these discussions are redundant, i.e. compilation is prevented regardless of the default settings. But I am curious about the optimal list lengths in cases where compilation occurs.

Answer 1

You should note that you are actually controlling the compiling and the array packing is just coupled to that and AFAIK can't be controlled independently (anymore). You can verify this with e.g. this uncompilable table body which generates the same result:

Developer`PackedArrayQ[Table[i /. x_ /; x > 300 :> RandomReal[], {i, 1, 251}]]
False

I would expect that the compiling dominates over the array packing concerning runtime overhead and thus that the dependence on the actual body of the table is much stronger than that of the system you are on. If that expectation isn't completely wrong an optimization with regards to the system might be rather useless. Here are examples to demonstrate this:

SetSystemOptions["CompileOptions" -> {"TableCompileLength" -> \[Infinity]}];
uncompiled = Map[
   Function[x, Timing[Do[With[{y = RandomReal[]},
         Table[
          Abs[i - y]/(Exp@Sin[y*i]*i^2 + 1 - 0.5*(i + y)^23), {i, x}]
         ], {1000}];][[1]]],
   Range[1, 50]
   ];

SetSystemOptions["CompileOptions" -> {"TableCompileLength" -> 1}];
compiled = Map[
   Function[x, Timing[Do[With[{y = RandomReal[]},
          Table[
           Abs[i - y]/(Exp@Sin[y*i]*i^2 + 1 - 0.5*(i + y)^23), {i, x}]
          ], {1000}];][[1]]],
    Range[1, 50]
   ];

ListLinePlot[{compiled, uncompiled}]

SetSystemOptions["CompileOptions" -> {"TableCompileLength" -> \[Infinity]}];
uncompiled = Map[
    Function[x, Timing[Do[
        With[{y = RandomReal[]}, Table[i + y, {i, x}]],
        {5000}];][[1]]],
   Range[60, 180]
   ];

SetSystemOptions["CompileOptions" -> {"TableCompileLength" -> 1}];
compiled = Map[
   Function[x, Timing[Do[
        With[{y = RandomReal[]}, Table[i + y, {i, x}]],
        {5000}];][[1]]],
   Range[60, 180]
  ];

ListLinePlot[{compiled, uncompiled}]

If you compare the two plots you will see that for these two cases the optimal compile length is very different.

To get most out of your system, you would need to adopt these settings to the problem at hand (and of course change them for another).

This of course assumes that you are talking about runtime efficiency, although array packing is just as well (or even in the first place) about memory efficiency.

Answer 2

Here is what I am doing using Table as an example but the same applies for Map, Fold, etc. To examine Table I map numbers onto it so to avoid any autocompilation issues with Map overlapping I set

SetSystemOptions["CompileOptions" -> "MapCompileLength" -> \[Infinity]];

First of all have a look at memory:

tmp1 = (SetSystemOptions["CompileOptions" -> "TableCompileLength" -> #];
     ByteCount[Table[i, {i, 1, #}]]) & /@ Range[1, 300];

SetSystemOptions["CompileOptions" -> "TableCompileLength" -> 250];
tmp2 = ByteCount[Table[i, {i, 1, #}]] & /@ Range[1, 300];

ListLinePlot[{tmp1, tmp2}]

So that looks as you'd expect. Now check timings:

tmp3 = ((SetSystemOptions["CompileOptions" -> "TableCompileLength" -> #1]; 
      Timing[Table[i, {i, 1, #1}];]) &) /@ Range[1, 500];

SetSystemOptions["CompileOptions" -> "TableCompileLength" -> \[Infinity]];
tmp4 = (Timing[Table[i, {i, 1, #1}];] &) /@ Range[1, 500];

ListLinePlot[{tmp3[[All, 1]], tmp4[[All, 1]]}]

On my system -- which fortuitously is a 5 year old slow computer -- these lines intersect somewhere between 280 and 290. I had considered adding a Do loop but a problem (for this analysis) arises:

tmp5 = (SetSystemOptions["CompileOptions" -> "TableCompileLength" -> #];
     Timing[Do[Table[i, {i, 1, #}];, {10}]]) & /@ Range[1, 500];

SetSystemOptions["CompileOptions" -> "TableCompileLength" -> \[Infinity]];
tmp6 = Timing[Do[Table[i, {i, 1, #}];, {10}]] & /@ Range[1, 500];

ListLinePlot[{tmp5[[All, 1]], tmp6[[All, 1]]}]

So looping 10 times reduces the point of intersection by ~10. I couldn't figure out a way around this so have not used Do.

The table is creating a list integers, i.e. returning the is. You would normally use Range for something like this. So the next test is to compare the timings when you do something with the is.

tmp7 = (SetSystemOptions["CompileOptions" -> "TableCompileLength" -> #];
     Timing[Table[i // N, {i, 1, #}];]) & /@ Range[1, 500];

SetSystemOptions["CompileOptions" -> "TableCompileLength" -> \[Infinity]];
tmp8 = Timing[Table[i // N, {i, 1, #}];] & /@ Range[1, 500];

ListLinePlot[{tmp7[[All, 1]], tmp8[[All, 1]]}]

We see now that the optimal length seems to be just over 120. After doing some other calculation with i in other tests, the "optimal" length seemed to vary from 50 through to 120. So it seems that there is no "exact" list length that is optimal, it is dependent on what is being done to the table iterator, however, on my system at least, the default autocompilation length can probably be set much lower for Table. (ditto Map etc.)

Some operations prevent compilation, e.g. Albert's example of a rule replacement:

tmp9 = ((SetSystemOptions["CompileOptions" -> "TableCompileLength" -> #1]; 
      Timing[Table[i /. x_ -> 1, {i, 1, #1}];]) &) /@ Range[1, 500];

SetSystemOptions["CompileOptions" -> "TableCompileLength" -> \[Infinity]];
tmp10 = (Timing[Table[i /. x_ -> 1, {i, 1, #1}];] &) /@ Range[1, 500];

ListLinePlot[{tmp9[[All, 1]], tmp10[[All, 1]]}]

But by definition the default value for the list lengths only has meaning if compilation takes place.

So this is one way of determining what value to use for list lengths for autocompilations (that is all based on what I think the autocompilation number represents). There may be other ways of doing this.