6
$\begingroup$

I am creating a random 3d data set in Matematica 12.1. Then I am selecting all points that are in a certain range of one axis.

The same I am doing in Python (same computer, Python 3.8.5, numpy 1.19.2)

RESULT: It seems that Python is able to select much faster (1.7 sec) than Mathematica (5.2 sec). What is the reason for that? For selection in Mathematica I used the fastest solution, which is by Carl Woll (see here at bottom).

SeedRandom[1];
coordinates = RandomReal[10, {100000000, 3}];

selectedCoordinates = 
   Pick[coordinates, 
    Unitize@Clip[coordinates[[All, 1]], {6, 7}, {0, 0}], 
    1]; // AbsoluteTiming

{5.16326, Null}

Dimensions[coordinates]

{100000000, 3}

Dimensions[selectedCoordinates]

{10003201, 3}

PYTHON CODE:

import time
import numpy as np
 
np.random.seed(1)
coordinates = np.random.random_sample((100000000,3))*10

start = time.time()
selectedCoordinates = coordinates[(coordinates[:,0] > 6) & (coordinates[:,0] < 7)]
end = time.time()

print(end-start)

print(coordinates.shape)

print(selectedCoordinates.shape)

1.6979997158050537

(100000000, 3)

(9997954, 3)
$\endgroup$
2
  • 3
    $\begingroup$ Is your ultimate goal to select or to count? $\endgroup$ – yarchik Feb 19 at 23:34
  • $\begingroup$ It is to select. $\endgroup$ – mrz Feb 20 at 10:18
6
$\begingroup$

I would argue that it's not a fair comparison unless you consider the performance with NumericArray, as the default Mathematica list-of-lists has many other features regarding numeric stability, etc. that are not present in a merely list of Real32 numbers.

Let me demonstrate: In your code you do something like:

SeedRandom[1];
coordinates = RandomReal[10, {100000000, 3}];

Clear[f]
f[coordinates_] := (Pick[coordinates, 
     Unitize@Clip[coordinates[[All, 1]], {6, 7}, {0, 0}], 1];) // 
  AbsoluteTiming

f[coordinates] (*4.0478 s on my MacBook*)

Now let's convert coordinates into a NumericArray (similar to what is being used internally in numpy):

coordinates32 = NumericArray[coordinates, "Real32"];
f[coordinates32] (* 1.09 s on my MacBook *)

This gives us a 3.7x speedup, comparable to the performance gains you observe in Python.

$\endgroup$
3
  • 4
    $\begingroup$ In the end, 5.2 sec is an excellent performance as the elements could be numbers, but also complex numbers, symbols, functions, etc. Try to do this with Python with no change in your code! $\endgroup$ – Denis Cousineau Feb 20 at 22:30
  • 1
    $\begingroup$ Exactly. @deniscousineau speaks my mind. $\endgroup$ – Joshua Schrier Feb 20 at 23:19
  • $\begingroup$ Thank you for the explanation. $\endgroup$ – mrz Mar 1 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.