Selecting real pairs in a data set

Question

Suppose we have a list of data, say,

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}}

I want to select a subset which includes only real pairs:

{{1, 3}, {-5, 0}, {3, 6}}

If I use Re[data],it also gives {2, 1},which I don' t want it. Is there any way to select those pairs for which both members are pure real? I need a program which works for long lists. In fact, I need a long list of data for ListDensityPlot. If any complex number is left, I won't be able to plot.

Answer 1

You can use FreeQ and Complex, which will search each element of the list

Select[data, FreeQ[#, Complex] &]

{{1, 3}, {-5, 0}, {3, 6}}

Answer 2

Here is a fast version:

Cases[data, {_Real | _Integer, _Real | _Integer}]

{{1, 3}, {-5, 0}, {3, 6}}

Benchmark

Using Mr.Wizard's benchmarking code:

big = Join[RandomReal[{-9, 9}, {5000, 2}], 
    RandomComplex[{-2 - I, 5 + 3 I}, {3000, 2}]] // RandomSample;

(*jjc385*) Select[big, And @@ Internal`RealValuedNumberQ /@ # &] // Length // RepeatedTiming

(*Mr.Wizard*) getReal = Pick[#, Unitize[Abs@Im@#.{1, 1}], 0] &;
    getReal[big] // Length // RepeatedTiming

(*UnchartedWorks*) (big // Select[# ∈ Reals &]) // Length // RepeatedTiming

(*KraZug*) Select[big, FreeQ[#, Complex] &] // Length // RepeatedTiming

{0.016, 5000}

{0.0085, 5000}

{0.0084, 5000}

{0.0076, 5000}

This method is about 7 times faster:

Cases[big, {_Real | _Integer, _Real | _Integer}] // Length // RepeatedTiming

{0.00109, 5000}

Update: benchmarks of updated method of Mr. Wizard

For pure numerical data (which can be converted to PackedArray), Mr. Wizard's method is twice faster than mine:

(*Mr.Wizard*)
Developer`ToPackedArray[getReal[Developer`ToPackedArray[big, Complex]], Real] // Length // RepeatedTiming

{0.00055, 5000}

For data which are already in complex PackedArray form, Mr. Wizard's results are 5 times faster:

(*Mr.Wizard*)
big2 = Developer`ToPackedArray[big, Complex];
Developer`ToPackedArray[getReal[big2], Real] // Length // RepeatedTiming

{0.00021, 5000}

Conclusion

Which filtering method is the fastest depends on the contents of your data.

If there are symbols (e.g. Missing) inside your data, then my solution is faster.

If data consists only from numbers (as in your example), it is faster to convert it to the PackedArray form and use Mr.Wizards solution.

Answer 3

Quick and dirty:

Select[data, And @@ Internal`RealValuedNumberQ /@ # &]

{{1, 3}, {-5, 0}, {3, 6}}

or, similarly but without a pure function:

Select[data, Apply[And] @* Map[Internal`RealValuedNumberQ] ]

---

I was somewhat surprised to find Mathematica is lacking a function RealQ to test whether a number is real. This question asked about this very thing, and I refer you to its answers, which explain how to implement such a function realQ on your own. One example is

ClearAll[realQ]

realQ[x_List] := And @@ realQ /@ x
realQ[x_] := Internal`RealValuedNumberQ[x]

Then you can simplify the task at hand to

Select[data, realQ]

{{1, 3}, {-5, 0}, {3, 6}}

Notice that this was much simpler from the realQ[x_List]:= ... definition, which says realQ acting on a list is only true if all of the elements of the list are real.

Answer 4

Not nearly as clean as KraZug's method but a bit faster, if it matters

getReal = Pick[#, Unitize[Abs@Im@#.{1, 1}], 0] &;

getReal[data]

{{1, 3}, {-5, 0}, {3, 6}}

Timings:

big = Join[RandomReal[{-9, 9}, {5000, 2}], 
    RandomComplex[{-2 - I, 5 + 3 I}, {3000, 2}]] // RandomSample;

Select[big, And @@ Internal`RealValuedNumberQ /@ # &] // Length // RepeatedTiming
Select[big, FreeQ[#, Complex] &]                      // Length // RepeatedTiming
getReal[big]                                          // Length // RepeatedTiming

{0.017, 5000}

{0.0097, 5000}

{0.0072, 5000}

---

Packed arrays

I chose Pick because it is fast on packed arrays, forgetting that the format I chose for big would not be packed. This makes my method significantly less competitive as seen in Shadowray's answer. However there is a case where it is much faster, even if that case now seems a little contrived. If we force packing of big as Complex data getReal demonstrates the performance I was aiming for. This can be followed by conversion to packed Real to clear out the 0. I terms. On the question example that looks like this:

Developer`ToPackedArray[data, Complex];
getReal[%]
Developer`ToPackedArray[%, Real]

{{1. + 0. I, 3. + 0. I}, {-5. + 0. I, 0. + 0. I}, {3. + 0. I, 6. + 0. I}}

{{1., 3.}, {-5., 0}, {3., 6.}}

If the ultimate target of this filtering is ListDensityPlot this format will be fine.

I cannot directly compare the performance of getReal on this packed Complex to other methods such as Select[FreeQ[Complex]] and Cases[data, {_Real | _Integer, _Real | _Integer}] as these won't work on the packed data. I shall therefore compare their performance on unpacked big to getReal on the packed form I'll name big2.

big2 = Developer`ToPackedArray[big, Complex];

Select[big, FreeQ[#, Complex] &]                  // Length // RepeatedTiming
Cases[big, {_Real | _Integer, _Real | _Integer}]  // Length // RepeatedTiming
Developer`ToPackedArray[getReal[big2], Real]      // Length // RepeatedTiming

{0.0162, 5000}

{0.01, 5000}

{0.00114, 5000}

{0.00019, 5000}

This doesn't include the initial conversion to packed Complex, which itself takes:

Developer`ToPackedArray[big, Complex]; // RepeatedTiming

{0.00029, Null}

So with that overhead my code is 114 / (19 + 29) ~= 2.3 times faster than Shadowray's next-fastest code, and without that overhead it is almost four times faster.

Answer 5

In:

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}};
data // Select[# ∈ Reals &]

Out:

{{1, 3}, {-5, 0}, {3, 6}}

If you use FreeQ[#,Complex]&, I don't think it's correct. I think the method with the fewest assumptions should be selected. For instance.

In:

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {x, y}}
data // Select[# ∈ Reals &]
Select[data, FreeQ[#, Complex] &]

Out:

{{1, 3}, {-5, 0}, {3, 6}}

{{1, 3}, {-5, 0}, {3, 6}, {x, y}}

Answer 6

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}};
Cases[data, {(_Integer|_Real)..}]
(* {{1, 3}, {-5, 0}, {3, 6}} *)

Answer 7

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {Pi, E}};

Since V 13.3 we have RealValuedNumberQ and RealValuedNumericQ :

Cases[data, {__?RealValuedNumberQ}]

{{1, 3}, {-5, 0}, {3, 6}}

Cases[data, {__?RealValuedNumericQ}]

{{1, 3}, {-5, 0}, {3, 6}, {Pi, E}}

Answer 8

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {π, E}}; (* Thanks to  \@eldo *)

Using AllTrue:

Select[AllTrue[Head@# =!= Complex &]][data]

Using Except:

Cases[data, {Except[_Complex] ..}]

---

Result:

{{1, 3}, {-5, 0}, {3, 6}, {π, E}}