1
$\begingroup$

I have three columns - P1,P2,M and 411935 rows. I want to extract only those rows which satisfy P1>=P2>=0.25*M. How can I do it?

$\endgroup$
0
2
$\begingroup$
Select[yourTable, #[[1]]>=#[[2]]>=(0.25*#[[3]])&]
$\endgroup$
0
2
$\begingroup$

If runtime is a major concern to you, you could use this Pick along with vectorized integer arithmetic instead:

yourTable = RandomReal[{-1, 1}, {411935, 3}];
aa = Select[
      yourTable, 
      #[[1]] >= #[[2]] >= (0.25*#[[3]]) &
      ]; // RepeatedTiming // First
bb = Pick[
      yourTable,
      Times@@UnitStep[Differences[{0.25,1.,1.} Transpose[yourTable][[{3,2,1}]]]],
      1
      ]; // RepeatedTiming // First
aa == bb

0.76

0.0142

True

If your columns are already given as vector P1, P2, and M, then you can skip the costly transposition to get the transposed result of the above:

{P1, P2, M} = Transpose[yourTable];
(
   picker = UnitStep[Subtract[P1, P2]] UnitStep[Subtract[P2, 0.25 M]];
   {nP1, nP2, nM} = Map[Pick[#, picker, 1] &, {P1, P2, M}];
   ) // RepeatedTiming // First

{nP1, nP2, nM} == Transpose[bb]

0.0075

True

$\endgroup$
2
$\begingroup$

Here's a variation of @Henrik's Pick approach that avoids Transpose:

cc = Pick[
    yourTable,
    UnitStep[yourTable . {{1,0}, {-1,1}, {0,-.25}}] . {1,1},
    2
]; //RepeatedTiming

{0.010, Null}

Compare to his answer:

bb = Pick[
    yourTable,
    Times@@UnitStep[Differences[{0.25,1.,1.} Transpose[yourTable][[{3,2,1}]]]],
    1
]; // RepeatedTiming

bb === cc

{0.020, Null}

True

$\endgroup$
3
  • $\begingroup$ Ah, very good improvement, indeed! In particular, I pretty much like how you replaced Times by a Dot. Plus, this is much short to type.^^ $\endgroup$ Aug 31 '18 at 11:34
  • $\begingroup$ @HenrikSchumacher Exactly. I saw those Times operations, and thought figuring out how to use Dot instead should provide a nice speed up. $\endgroup$
    – Carl Woll
    Aug 31 '18 at 11:39
  • $\begingroup$ Surprisingly, I found that picker = UnitStep[P1 - P2] + UnitStep[P2 - 0.25 M]; {nP1, nP2, nM} = Map[Pick[#, picker, 2] &, {P1, P2, M}]; is not faster than picker = UnitStep[P1 - P2] UnitStep[P2 - 0.25 M]; {nP1, nP2, nM} = Map[Pick[#, picker, 1] &, {P1, P2, M}]; on my machine. But anyways, the Dot saves a transposition. $\endgroup$ Aug 31 '18 at 11:49
1
$\begingroup$
data = Pick[DATA, 
  Table[p1[[i]] >= p2[[i]] >= 0.25*M[[i]], {i, 411935}]]

Thank you all for sharing your ideas. I used the above by combining your ideas and it worked.

$\endgroup$

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .