4
$\begingroup$

The problem of Mathematica is there are too many ways to perform one task. That leads to confusion for new users because the performances of these methods are so different.

I'm learning the best coding style to gain performance in Mathematica. However, there is something that I don't understand and need the explication of the experts.


Let takes this simple example. I have a list of data consist of 10^7 elements {{x1,y1},{x2,y2},...}. Now I want to make a list like {{x1,f[y1]},{x2,f[y2]},...}. For this task, there are about 13 methods.

list = Table[{i, 2 i}, {i, 1., 10^7}];

This is the packed array:

Developer`PackedArrayQ[list]
On["Packing"];

And here is the performance of these 13 methods. The performance difference between these methods is huge. (from 0.5s to 20s). The time is measured by AbsoluteTiming, the f function is Sin.

enter image description here


Here are my questions:

  • Q1: Let's compare methods #11 vs #10. Why using only one built-in function MapAt is much slower than {Copy the whole list, apply function f to the Part, and recall the new list}? In general, using 1 Function is faster than combine several functions.
  • Q2: Between the #9, and #10. Again, using MapAt, Pure function, and Map is faster than using MapAt with a built-in position option. Why? What is the purpose of implementing the position option in MapAt when this method is slower than combining other existing functions?
  • Q3: Compare #3, and #4, using the same function Table, but with a different approach, the performance is 3 times slowers. Should Mathematica eliminate the method that is not optimized?
  • Q4: Compare 6 and 4. Method 6 is the Functional Style with Apply, Pure Function. Method 4 is a Procedural way and breaking the list into each element. Why the functional style which treats the data as the whole list is much slower than the naive Procedural style?
  • A1: Unpacking verify is very useful.

So in general, which is your preferred coding style for performance? I would like to learn.


In[138]:= $Version

Out[138]= "12.0.0 for Microsoft Windows (64-bit) (April 6, 2019)"

Here is the code to verify:

list = Table[{i, 2 i}, {i, 1., 10^7}];
Developer`PackedArrayQ[list]
On["Packing"];


timelist = {
   Transpose[{#[[1]], Sin[#[[2]]]}] &@Transpose@list; // 
    AbsoluteTiming,
   Transpose[{list[[All, 1]], Sin[list[[All, 2]]]}]; // AbsoluteTiming,
   Table[{i[[1]], Sin[i[[2]]]}, {i, list}]; // AbsoluteTiming,
   Table[{list[[i, 1]], Sin[list[[i, 2]]]}, {i, 1, Length[list]}]; // 
    AbsoluteTiming,
   {#[[1]], Sin[#[[2]]]} & /@ list; // AbsoluteTiming,
   {#1, Sin[#2]} & @@@ list; // AbsoluteTiming,
   MapThread[{#1, Sin[#2]} &, Transpose[list]]; // AbsoluteTiming,
   Inner[#1[#2] &, {# &, Sin[#] &}, Transpose[list], List]; // 
    AbsoluteTiming,
   MapAt[Sin, #, 2] & /@ list; // AbsoluteTiming,
   MapAt[Sin, list, {All, 2}]; // AbsoluteTiming,
   AbsoluteTiming[list2 = list; 
    list2[[All, 2]] = Sin[list2[[All, 2]]]; list2;],
   Map[Composition[
       Through, {Composition[Identity, First], 
        Composition[Sin, Last]}], list]; // AbsoluteTiming,
   list /. {x_, y_} -> {x, Sin[y]}; // AbsoluteTiming
   };

funclist = ToString /@ {
    HoldForm[Transpose[{#[[1]], Sin[#[[2]]]}] &@Transpose@list],
    HoldForm[Transpose[{list[[All, 1]], Sin[list[[All, 2]]]}]],
    HoldForm[Table[{i[[1]], Sin[i[[2]]]}, {i, list}]],
    HoldForm[
     Table[{list[[i, 1]], Sin[list[[i, 2]]]}, {i, 1, Length[list]}]],
    HoldForm[{#[[1]], Sin[#[[2]]]} & /@ list],
    HoldForm[{#1, Sin[#2]} & @@@ list],
    HoldForm[MapThread[{#1, Sin[#2]} &, Transpose[list]]],
    HoldForm[Inner[#1[#2] &, {# &, Sin[#] &}, Transpose[list], List];],
    HoldForm[MapAt[Sin, #, 2] & /@ list],
    HoldForm[MapAt[Sin, list, {All, 2}]],
    HoldForm[list2 = list; list2[[All, 2]] = Sin[list2[[All, 2]]]; 
     list2],
    HoldForm@
     Map[Composition[
       Through, {Composition[Identity, First], 
        Composition[Sin, Last]}], list],
    HoldForm[list /. {x_, y_} -> {x, Sin[y]}]
    };

unpack = {"No", "No", "No", "No", "No", "Yes", "Yes", "Yes", "No", 
    "Yes", "No", "Yes", "Yes"};;
sol = SortBy[
   Transpose[{Range@Length@funclist, funclist, unpack, 
     timelist[[All, 1]]}], Last];
Grid[PrependTo[sol, {"#", "Method", "Unpacking", "AbsoluteTiming"}], 
 Frame -> All]
$\endgroup$
8
  • $\begingroup$ I don't understand what you mean @chris $\endgroup$
    – Nam Nguyen
    Nov 2, 2020 at 13:36
  • 1
    $\begingroup$ In general, you want to do operations on whole arrays as much as possible. Adding arrays; dotting them; using Listable functions (like Sin) on the array directly. That sort of thing. That way you ensure the least amount of high-level overhead. $\endgroup$ Nov 2, 2020 at 13:50
  • 1
    $\begingroup$ I think most of your questions can be answered by considering array unpacking: See e.g. this question and related. You can use On["Packing"] and run the examples from your question to see which examples unpack how much $\endgroup$
    – Lukas Lang
    Nov 2, 2020 at 13:51
  • 5
    $\begingroup$ Also, please include the code for your tests in the question, so others can play around with it without having to enter everything manually $\endgroup$
    – Lukas Lang
    Nov 2, 2020 at 13:52
  • 2
    $\begingroup$ BTW, Transpose[{#[[1]], Sin[#[[2]]]}] &@Transpose@list is probably faster than any of your alternatives. $\endgroup$
    – Michael E2
    Nov 2, 2020 at 14:59

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy