6
$\begingroup$

Consider the following list:

tabb = RandomReal[{0.0000001, 10^5}, {10^6, 80}];

I want to extract, e.g., the columns from 1 to 20. It is reasonably fast:

tabb[[All, Range[1, 20]]]; // AbsoluteTiming // First

0.077998

However, if I first make the following replacement,

tabb[[All, {2}]] = tabb[[All, {22}]] = Table[{i}, {i, 1, Length[tabb], 1}];

it starts working much slower:

tabb[[All, Range[1, 20]]]; // AbsoluteTiming // First

0.426746

Twe two questions I have are:

  1. Is it possible to speed up extraction of the columns compared to the first example?
  2. What is the reason for slowdown in the second example?
$\endgroup$
1
  • 1
    $\begingroup$ There is also On[["Packing"]] (see here) which will produce a message of the type Developer`FromPackedArray::punpack: Unpacking array with dimensions {1000000,1} in call to System`Private`PackedList. if a packed array is unpacked. $\endgroup$
    – user1066
    Commented Oct 27, 2023 at 15:35

2 Answers 2

9
$\begingroup$

what is the reason for slowdown for the second example?

It got unpacked, that is why

tabb=RandomReal[{0.0000001,10^5},{10^6,80}];
Developer`PackedArrayQ[tabb]

Mathematica graphics

tabb[[All, {2}]] = tabb[[All, {22}]] = Table[{i}, {i, 1, Length[tabb], 1}];
Developer`PackedArrayQ[tabb]

Mathematica graphics

Packed array are much faster.

$\endgroup$
7
$\begingroup$

The reason of the slowdown is the mixture of reals with integers.

tabb = RandomReal[{0.0000001, 10^5}, {10^6, 80}];

tabb[[All, Range[1, 20]]]; // AbsoluteTiming // First

0.06858

Notice the 1. instead of 1:

tabb[[All, {2}]] = 
  tabb[[All, {22}]] = Table[{i}, {i, 1., Length[tabb], 1}];

tabb[[All, Range[1, 20]]]; // AbsoluteTiming // First

0.066825

Thanks to Nasser:

Developer`PackedArrayQ[tabb]

True

$\endgroup$
2
  • $\begingroup$ Can you clarify whether or not the lists remain packed using this example? $\endgroup$ Commented Oct 26, 2023 at 11:12
  • 1
    $\begingroup$ Using the same test as Nasser, one can see that tabb remains packed - see updated answer. $\endgroup$
    – eldo
    Commented Oct 26, 2023 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.