# How to write this pattern in Pick function

Suppose I have a list called mask composed of 1,2,3,...n. n is different in different situation. Let me takes n=3 for demonstration

mask=RandomInteger[{1,3},1000000]


and another list

list = RandomReal[{0, 1}, 1000000];


I want to pick those element corresponding not equal to 1.

Pick[list, mask, _?(# != 1 &)]; // Timing


This takes 1.125 sec

But If I already know mask only composed of 1,2,3, then this

Pick[list, mask, 2 | 3]; // Timing


is faster, it takes 0.25 sec

But the problem is I am not sure that is in mask, so this is not general.

So the question is there more efficient way than this _?(# != 1 &) pattern? Why is it slower then pattern 2|3?

• As for why 2|3 is faster than _?(# != 1 &): it's because the latter involves evaluating Mathematica code (evaluating the pure function) for each test. The former doesn't. Oct 25, 2015 at 17:25
• I tried Pick[list, mask, Except] but it fails because the Except matches the whole list. Pick[list, mask, Except[1, _Integer]] works and is the same speed as 2 | 3. Oct 25, 2015 at 23:49
• @Szabolcs I don't understand. 2|3 doesn't evaluate for each test? Then how can it know which one to pick? Oct 26, 2015 at 0:30
• @2012rcampion good observation! Thank you! But in this particular case, Szabolcs's method is faster. Oct 26, 2015 at 5:57
• @matheorem Think about how Pick might be implemented in C. For _?(# != 1 &) you'd need a callback to the main evaluator (i.e. run Mathematica code) for each test. For 2|3 you don't. You just need to test for equality between 2 (or 3) and the given list element, but this test doesn't involve running Mathematica code. It can be done only in C. Oct 26, 2015 at 7:02

Since I think version 8, Pick is optimized for the case when the pattern is a single element (i.e. 1 or 2 but not 1|2), and when the inputs are packed arrays.

If you need performance, make sure that you hit this special case. Use vectorized arithmetic operations to transform the lists into a suitable form.

Pick[list, mask, _?(# != 1 &)]; // AbsoluteTiming
(* {0.547087, Null} *)

Pick[list, Unitize[mask - 1], 1]; // AbsoluteTiming
(* {0.019021, Null} *)


My BoolEval package tries to automate this process for more complicated cases, at the cost of only a little performance.

<< BoolEval

BoolPick[list, mask == 1]; // AbsoluteTiming
(* {0.029157, Null} *)

• Thank you so much. What do you mean by vectorized arithmetic operations, why is it so fast? Except Unitize, are there other vectorized arithmetic operations` useful? Oct 26, 2015 at 0:36
• @matheorem The rule of thumb is that arithmetic on packed arrays of machine numbers tends to be fast. That's because it can be implemented very efficiently in terms of SIMD instructions, it's easy to parallelize (think adding two arrays) and packed array storage is very efficient. Oct 26, 2015 at 7:24