1
$\begingroup$

I have three columns - P1,P2,M and 411935 rows. I want to extract only those rows which satisfy P1>=P2>=0.25*M. How can I do it?

$\endgroup$
0
2
$\begingroup$
Select[yourTable, #[[1]]>=#[[2]]>=(0.25*#[[3]])&]
$\endgroup$
0
2
$\begingroup$

If runtime is a major concern to you, you could use this Pick along with vectorized integer arithmetic instead:

yourTable = RandomReal[{-1, 1}, {411935, 3}];
aa = Select[
      yourTable, 
      #[[1]] >= #[[2]] >= (0.25*#[[3]]) &
      ]; // RepeatedTiming // First
bb = Pick[
      yourTable,
      Times@@UnitStep[Differences[{0.25,1.,1.} Transpose[yourTable][[{3,2,1}]]]],
      1
      ]; // RepeatedTiming // First
aa == bb

0.76

0.0142

True

If your columns are already given as vector P1, P2, and M, then you can skip the costly transposition to get the transposed result of the above:

{P1, P2, M} = Transpose[yourTable];
(
   picker = UnitStep[Subtract[P1, P2]] UnitStep[Subtract[P2, 0.25 M]];
   {nP1, nP2, nM} = Map[Pick[#, picker, 1] &, {P1, P2, M}];
   ) // RepeatedTiming // First

{nP1, nP2, nM} == Transpose[bb]

0.0075

True

$\endgroup$
2
$\begingroup$

Here's a variation of @Henrik's Pick approach that avoids Transpose:

cc = Pick[
    yourTable,
    UnitStep[yourTable . {{1,0}, {-1,1}, {0,-.25}}] . {1,1},
    2
]; //RepeatedTiming

{0.010, Null}

Compare to his answer:

bb = Pick[
    yourTable,
    Times@@UnitStep[Differences[{0.25,1.,1.} Transpose[yourTable][[{3,2,1}]]]],
    1
]; // RepeatedTiming

bb === cc

{0.020, Null}

True

$\endgroup$
3
  • $\begingroup$ Ah, very good improvement, indeed! In particular, I pretty much like how you replaced Times by a Dot. Plus, this is much short to type.^^ $\endgroup$ – Henrik Schumacher Aug 31 '18 at 11:34
  • $\begingroup$ @HenrikSchumacher Exactly. I saw those Times operations, and thought figuring out how to use Dot instead should provide a nice speed up. $\endgroup$ – Carl Woll Aug 31 '18 at 11:39
  • $\begingroup$ Surprisingly, I found that picker = UnitStep[P1 - P2] + UnitStep[P2 - 0.25 M]; {nP1, nP2, nM} = Map[Pick[#, picker, 2] &, {P1, P2, M}]; is not faster than picker = UnitStep[P1 - P2] UnitStep[P2 - 0.25 M]; {nP1, nP2, nM} = Map[Pick[#, picker, 1] &, {P1, P2, M}]; on my machine. But anyways, the Dot saves a transposition. $\endgroup$ – Henrik Schumacher Aug 31 '18 at 11:49
1
$\begingroup$
data = Pick[DATA, 
  Table[p1[[i]] >= p2[[i]] >= 0.25*M[[i]], {i, 411935}]]

Thank you all for sharing your ideas. I used the above by combining your ideas and it worked.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.