Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have a list of the form:

testList = Table[{Flatten[RandomReal[{-200, 200}, {1, 2}], 1], RandomReal[{-200, 200}, {69, 2}]}, {i, 1, 100}];

And I would like to perform the following rescaling operation very quickly:

Rescale[testList, {-160 Sqrt[5], 160 Sqrt[2]}] // AbsoluteTiming

And indeed we see that this takes about $\approx 1$ millisecond.

However, if we manually specify testList:

riggedTestList = {{72.0003, 254.632}, {{0., 0.}, {164.523, -1.48019}, {-36.4013, -1.6529}, {-30.6006, -9.68836}, {79.1769, -9.38496}, {127.414, -13.3351}, {26.2267, -14.9839}, {101.471, -17.2131}, {112.759, -20.2044}, {140.786, -20.3125}, {152.117, -23.4485}, {171.028, -28.6424}, {128.855, -34.8548}, {67.2216, -48.2581}, {-58.5145, -53.227}, {147.275, -52.8399}, {-2.11577, -54.7374}, {160.367, -57.3066}, {-43.8697, -59.9146}, {50.3993, -63.4993}, {105.478, -64.6626}, {-38.5779, -73.1765}, {167.692, -74.2231}, {38.1779, -75.0274}, {182.719, -75.3023}, {101.273, -76.3709}, {91.2674, -78.7477}, {140.126, -83.058}, {167.292, -83.068}, {-30.6985, -90.7144}, {73.0603, -91.5605}, {15.3827, -93.4644}, {168.049, -97.4066}, {101.349, -97.9227}, {148.875, -105.038}, {-27.7536, -106.1}, {119.991, -110.415}, {170.865, -110.703}, {-1.03466, -111.89}, {96.2519, -121.383}, {179.463, -127.087}, {-62.9256, -127.913}, {172.077, -131.731}, {-41.8538, -137.851}, {171.768, -143.748}, {28.4562, -146.02}, {71.0586, -145.51}, {149.767, -146.569}, {37.6894, -150.356}, {-26.1955, -159.44}, {-41.9501, -163.069}, {135.019, -164.213}, {-62.5847, -172.048}, {7.02217, -174.589}, {133.175, -175.744}, {0.392205, -184.99}, {56.418, -189.734}, {83.9584, -196.068}, {-9.36728, -198.523}, {141.564, -197.764}, {43.6112, -200.002}, {133.577, -201.431}, {16.5308, -203.89}, {79.8118, -207.938}, {178.305, -210.113}, {-45.2895, -216.227}, {-4.60035, -234.998}, {3.16025, -240.952}, {78.6896, -248.948}}};

riggedTestList = Table[riggedTestList, {i, 1, 100}];

And then attempt the same rescale on this array with the same dimensions as testList:

Rescale[riggedTestList, {-160 Sqrt[5], 160 Sqrt[2]}] // AbsoluteTiming

The Rescale operation takes $\approx 33$ milliseconds, a difference of over an order of magnitude. How could this be the case? Both Developer`PackedArrayQ[testList] and Developer`PackedArrayQ[riggedTestList] return False, so I don't think it's a list packing issue?

share|improve this question
add comment

1 Answer

up vote 10 down vote accepted

Actually, it is all about packing. By using RandomReal you an generating packed sub-arrays even if the complete array is not packed (and can't be, due to irregular shape):

Map[PackedArrayQ, testList, {2}] // Short
{{True,True},{True,True},{True,True},{True,True},{True,True},<<91>>,{True,True},
 {True,True},{True,True},{True,True}}

Let's look at some timings more closely:

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := Do[If[# > 0.3, Return[#/5^i]] & @@ Timing@Do[func, {5^i}], {i, 0, 15}]

Needs["Developer`"]

Rescale[testList, {-160 Sqrt[5], 160 Sqrt[2]}] // timeAvg
0.0005744
Rescale[riggedTestList, {-160 Sqrt[5], 160 Sqrt[2]}] // timeAvg
0.0256

Fully unpacking testList results in the same timing as riggedTestList:

unpacked = FromPackedArray @ testList;
Rescale[unpacked, {-160 Sqrt[5], 160 Sqrt[2]}] // timeAvg
0.0256

Packing riggedTestList in the same degree as testList also matches the faster timing:

repacked = Map[ToPackedArray, riggedTestList, {2}];
Rescale[repacked, {-160 Sqrt[5], 160 Sqrt[2]}] // timeAvg
0.0005744

Addressing a comment below, in version 7 under Windows the results are equivalent as expected:

Rescale[testList, {-160 Sqrt[5], 160 Sqrt[2]}] == 
  Rescale[unpacked, {-160 Sqrt[5], 160 Sqrt[2]}]

Rescale[riggedTestList, {-160 Sqrt[5], 160 Sqrt[2]}] == 
  Rescale[repacked, {-160 Sqrt[5], 160 Sqrt[2]}]

True

True

share|improve this answer
    
Ahh, I should have looked at the sublists. –  RM1618 Sep 23 '13 at 7:56
    
@RM1618 These things are fairly tricky. I didn't understand this stuff until someone else showed me. –  Mr.Wizard Sep 23 '13 at 7:57
    
Well thanks for passing along the knowledge. –  RM1618 Sep 23 '13 at 8:01
    
@RM1618 For that you're always welcome. :-) –  Mr.Wizard Sep 23 '13 at 8:02
    
Is it possible to have repacked actually look like Rescale[riggedTestList, {-160 Sqrt[5], 160 Sqrt[2]}] after rescaling? For me on version 9, I'm not seeing the Rescale operation performed? –  RM1618 Sep 23 '13 at 8:10
show 5 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.