# Performance of Select

I have a dataset of 3D coordinates with a length of about $$4\times 10^6$$.

From this volume I am sequentially selecting coordinates along one axis and manipulating this subset.

My question: Can the Select function be replaced by something that is faster.

Here is the example code with the needed time for selection:

SeedRandom[1];

coordinates = RandomReal[10, {4000000, 3}]; // AbsoluteTiming

{0.0989835, Null}

selectedCoordinates = Select[coordinates, #[[1]] > 6 && #[[1]] < 7 & ]; // AbsoluteTiming

{5.88215, Null}

Dimensions[selectedCoordinates]

{400416, 3}

• Pick[coordinates, 6 < # < 7 & /@ coordinates[[All, 1]]] is almost twice as fast as Select[..]
– kglr
Commented Oct 25, 2017 at 11:43
• You can compile your Select: compiled = Compile[{{coords, _Integer, 2}}, Select[coords, #[[1]] > 6 && #[[1]] < 7 &], CompilationTarget -> "C"] . Then compiled[coordinates] takes 0.2 secs on my machine. Commented Oct 25, 2017 at 11:51
• Cases[coordinates, {x_, y_, z_} /; x > 6 && y < 7] Assuming that you want to get #[[1]]>6 &&#[[2]]<7. Otherwise the output would always by {}. No integer can be >6 and <7 at the same time ,-).
– RMMA
Commented Oct 25, 2017 at 12:00
• @RMMA: Thank you for your remark. I changed to RandomReal.
– mrz
Commented Oct 25, 2017 at 14:30

res1 = Select[coordinates, #[[1]] > 6 && #[[1]] < 7 &]; //
AbsoluteTiming // First


6.997629

res2 = Select[coordinates, 6 < #[[1]] < 7 &]; // AbsoluteTiming // First


4.676356

res3 = Pick[coordinates, 6 < # < 7 & /@ coordinates[[All, 1]]]; //
AbsoluteTiming // First


5.266651

res4 = Pick[coordinates, (1 - UnitStep[# - 7]) (1 - UnitStep[6 - #]) &@
coordinates[[All, 1]], 1]; // AbsoluteTiming // First


0.353154

res6 = compiled[coordinates]; // AbsoluteTiming // First


0.667676

where

compiled = Compile[{{coords, _Real, 2}}, Select[coords, #[[1]] > 6 && #[[1]] < 7 &]]


is the method suggested in Leonid's comment (without the option CompilationTarget -> "C").

Equal[res1, res2, res3, res4, res5, res6]


True

• Thank so much. The last solution is my case about 35 times faster than Select.
– mrz
Commented Oct 25, 2017 at 14:04
• A fairer comparison would chain the inequalities for Select as well. And it would be nice to include the comparison for a compiled selector.
– Alan
Commented Oct 25, 2017 at 14:18
• @mrz, my pleasure, Thank you for the accept.
– kglr
Commented Oct 25, 2017 at 14:25
• @Alan, I added the variant of Select you suggested. I don't have a c compiler installed, so i cannot include timings for the method suggested by Leonid. Without the CompilationTarget->"C" compiled is slower than Pick.
– kglr
Commented Oct 25, 2017 at 14:32
• Something like Pick[c,UnitStep[c-6,7-c],1] is more compact, but is about 30 times slower than your res4 formulation! The problem is with the multi-dimensional UnitStep. Something like Pick[c, UnitStep[c - 6]*UnitStep[7 - c], 1] seems as fast or slightly faster than the res4 formulation. Commented Oct 25, 2017 at 16:14

Slightly faster than @kglr's solution is to use Clip:

SeedRandom[1];
coordinates = RandomReal[10, {4000000, 3}];

r1 = Pick[
coordinates,
Unitize @ Clip[coordinates[[All,1]], {6, 7}, {0, 0}],
1
];//RepeatedTiming

r2 = Pick[
coordinates,
(1-UnitStep[#-7]) (1-UnitStep[6-#])&@coordinates[[All,1]],
1
];//RepeatedTiming

r1 === r2


{0.10, Null}

{0.15, Null}

True

• Thanks a lot for your help ... I have to remenber Clip ... this function is extremely fast
– mrz
Commented Oct 25, 2017 at 19:08
• Thanks. I learn about Pick, combined with Clip How wise! Commented Oct 26, 2017 at 17:23

My question: can the Select function be replaced by something that is faster.

Yes! Check out the BoolEval package.

SeedRandom[1];
coordinates = RandomReal[10, {4000000, 3}]; // AbsoluteTiming
(* {0.118832, Null} *)

selectedCoordinates =
Select[coordinates, #[[1]] > 6 && #[[1]] < 7 &]; // AbsoluteTiming
(* {6.08899, Null} *)

Needs["BoolEval"]

selectedCoordinates2 = BoolPick[coordinates, 6 < coordinates[[All, 1]] < 7]; // AbsoluteTiming
(* {0.145518, Null} *)

selectedCoordinates == selectedCoordinates2
(* True *)
`

Be sure to read the documentation of the package to see more usage examples and learn about caveats.

• This is great! I just installed and tried it. Commented Aug 5, 2019 at 10:18