Performance of Select

Question

I have a dataset of 3D coordinates with a length of about $ 4\times 10^6 $.

From this volume I am sequentially selecting coordinates along one axis and manipulating this subset.

My question: Can the Select function be replaced by something that is faster.

Here is the example code with the needed time for selection:

SeedRandom[1];

coordinates = RandomReal[10, {4000000, 3}]; // AbsoluteTiming

{0.0989835, Null}

selectedCoordinates = Select[coordinates, #[[1]] > 6 && #[[1]] < 7 & ]; // AbsoluteTiming

{5.88215, Null}

Dimensions[selectedCoordinates]

{400416, 3}

Answer 1

res1 = Select[coordinates, #[[1]] > 6 && #[[1]] < 7 &]; // 
  AbsoluteTiming // First

6.997629

res2 = Select[coordinates, 6 < #[[1]] < 7 &]; // AbsoluteTiming // First

4.676356

res3 = Pick[coordinates, 6 < # < 7 & /@ coordinates[[All, 1]]]; // 
  AbsoluteTiming // First

5.266651

res4 = Pick[coordinates, (1 - UnitStep[# - 7]) (1 - UnitStep[6 - #]) &@
      coordinates[[All, 1]], 1]; // AbsoluteTiming // First

0.353154

res6 = compiled[coordinates]; // AbsoluteTiming // First

0.667676

where

compiled = Compile[{{coords, _Real, 2}}, Select[coords, #[[1]] > 6 && #[[1]] < 7 &]]`

is the method suggested in Leonid's comment (without the option `CompilationTarget -> "C").

Equal[res1, res2, res3, res4, res5, res6]

True

Answer 2

Slightly faster than @kglr's solution is to use Clip:

SeedRandom[1];
coordinates = RandomReal[10, {4000000, 3}];

r1 = Pick[
    coordinates,
    Unitize @ Clip[coordinates[[All,1]], {6, 7}, {0, 0}],
    1
];//RepeatedTiming

r2 = Pick[
    coordinates,
    (1-UnitStep[#-7]) (1-UnitStep[6-#])&@coordinates[[All,1]],
    1
];//RepeatedTiming

r1 === r2

{0.10, Null}

{0.15, Null}

True

Answer 3

My question: can the Select function be replaced by something that is faster.

Yes! Check out the BoolEval package.

SeedRandom[1];
coordinates = RandomReal[10, {4000000, 3}]; // AbsoluteTiming
(* {0.118832, Null} *)

selectedCoordinates = 
   Select[coordinates, #[[1]] > 6 && #[[1]] < 7 &]; // AbsoluteTiming
(* {6.08899, Null} *)

Needs["BoolEval`"]

selectedCoordinates2 = BoolPick[coordinates, 6 < coordinates[[All, 1]] < 7]; // AbsoluteTiming
(* {0.145518, Null} *)

selectedCoordinates == selectedCoordinates2
(* True *)

Be sure to read the documentation of the package to see more usage examples and learn about caveats.