If I read data from a data file and manipulate the list and do some calculations. The resulting is a list of dimensions:

{16, 59210}

Then I create a test list of the same dimensions:

test = RandomReal[{-2.0*10^6, 2.0*10^6}, {16, 60000}];

then if I do e.g.

Dimensions@δx // AbsoluteTiming
Dimensions@test // AbsoluteTiming
Head[δx[[1, 1]]] // AbsoluteTiming
Head[test[[1, 1]]] // AbsoluteTiming
δx[[1, 1]]; // AbsoluteTiming
test[[1, 1]]; // AbsoluteTiming
ByteCount@δx[[1, 1]] // AbsoluteTiming
ByteCount@test[[1, 1]] // AbsoluteTiming

{0.985994, {16, 59210}}
{5.*10^-6, {16, 100000}}
{0.997546, Real}
{4.*10^-6, Real}
{0.977426, Null}
{5.*10^-6, Null}
{1.03246, 16}
{5.*10^-6, 16}

I would like to figure out, what the difference is between the lists that the operations on them are so different performance wise. Both seem to be real, both have the same dimensions (in the case above, i even made the test list larger). I think it is due to the operations on the way to the δx and the list is unevaluated? - how do i verify this and how do I safely enforce evaluation then? - is Evaluate@δx enough (help does not specify, how it works lists)).

My ultimate goal is to have the test list of the same properties so that I can use it for performance analyses so all the operations on both lists should take the same amount of time.

  • 2
    $\begingroup$ RandomReal creates a packed array. I'm guessing that the process that reads the data file does not (i.e. Developer`PackedArrayQ[δx] === False). Operations on packed arrays are much quicker -- for example, determining the dimensions of an unpacked array involves scanning the entries whereas it is an intrinsic property of a packed array. $\endgroup$ – WReach Mar 10 '17 at 23:45
  • $\begingroup$ The time differences are extraordinary -- even more than I would expect for packed vs. unpacked. Also, something is not right about the benchmark data. Note, for example, how the dimensions are not all the same: {20, 60000} vs. {16, 59210} vs. {16, 100000}. Are sure that this is an apples-to-apples comparison? In any event, we will probably need more information to help diagnose this problem. $\endgroup$ – WReach Mar 10 '17 at 23:56
  • $\begingroup$ @WReach Sorry about the confusion with the dimensions but they are correct, the real data is {16, 59210} but I am simply testing with some rounded numbers {20, 60000} and {16, 100000}. $\endgroup$ – leosenko Mar 11 '17 at 0:11
  • $\begingroup$ Is it possible that δx has a delayed value (i.e. defined by δx := ...)? Check the definition by evaluating OwnValues[δx] // Short. $\endgroup$ – WReach Mar 11 '17 at 0:14
  • $\begingroup$ The first case probably has to run every element through the evaluator. Checking Length requires this in case there is a Sequence inside. $\endgroup$ – Daniel Lichtblau Mar 11 '17 at 17:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.