# Selecting real pairs in a data set

Suppose we have a list of data, say,

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}}


I want to select a subset which includes only real pairs:

{{1, 3}, {-5, 0}, {3, 6}}


If I use Re[data],it also gives {2, 1},which I don' t want it. Is there any way to select those pairs for which both members are pure real? I need a program which works for long lists. In fact, I need a long list of data for ListDensityPlot. If any complex number is left, I won't be able to plot.

• dataReal = DeleteCases[data, _?(! FreeQ[#, Complex] &)] Commented Jun 5, 2017 at 0:00

You can use FreeQ and Complex, which will search each element of the list

Select[data, FreeQ[#, Complex] &]

{{1, 3}, {-5, 0}, {3, 6}}

• In operator form: data // Select[FreeQ[Complex]] Commented Jun 4, 2017 at 6:32
• Given {{1, 3}, {-5, 0}, {3, 6}}, the result is correct, however I doubt "Non-Complex Number" is equivalent to "Real Number". It's not actually. Commented Jun 4, 2017 at 8:12
• @UnchartedWorks It is appropriate for the example input. Even Element fails in some cases as described here and linked in jjc385's answer. I love your concise code however (and I'm a little annoyed with myself I missed the opportunity to post that) and I just voted for your answer. Commented Jun 4, 2017 at 8:55
• @Mr.Wizard I made an assumption too, I assumed Element works, thanks for pointing it out that Element is not what I thought. Commented Jun 4, 2017 at 9:04

Here is a fast version:

Cases[data, {_Real | _Integer, _Real | _Integer}]


{{1, 3}, {-5, 0}, {3, 6}}

Benchmark

Using Mr.Wizard's benchmarking code:

big = Join[RandomReal[{-9, 9}, {5000, 2}],
RandomComplex[{-2 - I, 5 + 3 I}, {3000, 2}]] // RandomSample;

(*jjc385*) Select[big, And @@ InternalRealValuedNumberQ /@ # &] // Length // RepeatedTiming

(*Mr.Wizard*) getReal = Pick[#, Unitize[Abs@Im@#.{1, 1}], 0] &;
getReal[big] // Length // RepeatedTiming

(*UnchartedWorks*) (big // Select[# ∈ Reals &]) // Length // RepeatedTiming

(*KraZug*) Select[big, FreeQ[#, Complex] &] // Length // RepeatedTiming


{0.016, 5000}

{0.0085, 5000}

{0.0084, 5000}

{0.0076, 5000}

This method is about 7 times faster:

Cases[big, {_Real | _Integer, _Real | _Integer}] // Length // RepeatedTiming


{0.00109, 5000}

Update: benchmarks of updated method of Mr. Wizard

For pure numerical data (which can be converted to PackedArray), Mr. Wizard's method is twice faster than mine:

(*Mr.Wizard*)
DeveloperToPackedArray[getReal[DeveloperToPackedArray[big, Complex]], Real] // Length // RepeatedTiming


{0.00055, 5000}

For data which are already in complex PackedArray form, Mr. Wizard's results are 5 times faster:

(*Mr.Wizard*)
big2 = DeveloperToPackedArray[big, Complex];
DeveloperToPackedArray[getReal[big2], Real] // Length // RepeatedTiming


{0.00021, 5000}

Conclusion

Which filtering method is the fastest depends on the contents of your data.

If there are symbols (e.g. Missing) inside your data, then my solution is faster.

If data consists only from numbers (as in your example), it is faster to convert it to the PackedArray form and use Mr.Wizards solution.

• Nice improvement. (+1) Would you kindly time the additional code in my answer in version 11? Commented Jun 4, 2017 at 23:41
• @Mr.Wizard Thanks for an interesting update. The timings in version 11 are very close to what you got in v10. Your PackedArray solution is about 2 to 5 times faster then my pattern matching. Commented Jun 5, 2017 at 12:16

Quick and dirty:

Select[data, And @@ InternalRealValuedNumberQ /@ # &]

{{1, 3}, {-5, 0}, {3, 6}}


or, similarly but without a pure function:

Select[data, Apply[And] @* Map[InternalRealValuedNumberQ] ]


I was somewhat surprised to find Mathematica is lacking a function RealQ to test whether a number is real. This question asked about this very thing, and I refer you to its answers, which explain how to implement such a function realQ on your own. One example is

ClearAll[realQ]

realQ[x_List] := And @@ realQ /@ x
realQ[x_] := InternalRealValuedNumberQ[x]


Then you can simplify the task at hand to

Select[data, realQ]

{{1, 3}, {-5, 0}, {3, 6}}


Notice that this was much simpler from the realQ[x_List]:= ... definition, which says realQ acting on a list is only true if all of the elements of the list are real.

Not nearly as clean as KraZug's method but a bit faster, if it matters

getReal = Pick[#, Unitize[Abs@Im@#.{1, 1}], 0] &;

getReal[data]

{{1, 3}, {-5, 0}, {3, 6}}


Timings:

big = Join[RandomReal[{-9, 9}, {5000, 2}],
RandomComplex[{-2 - I, 5 + 3 I}, {3000, 2}]] // RandomSample;

Select[big, And @@ InternalRealValuedNumberQ /@ # &] // Length // RepeatedTiming
Select[big, FreeQ[#, Complex] &]                      // Length // RepeatedTiming
getReal[big]                                          // Length // RepeatedTiming

{0.017, 5000}

{0.0097, 5000}

{0.0072, 5000}


## Packed arrays

I chose Pick because it is fast on packed arrays, forgetting that the format I chose for big would not be packed. This makes my method significantly less competitive as seen in Shadowray's answer. However there is a case where it is much faster, even if that case now seems a little contrived. If we force packing of big as Complex data getReal demonstrates the performance I was aiming for. This can be followed by conversion to packed Real to clear out the 0. I terms. On the question example that looks like this:

DeveloperToPackedArray[data, Complex];
getReal[%]
DeveloperToPackedArray[%, Real]

{{1. + 0. I, 3. + 0. I}, {-5. + 0. I, 0. + 0. I}, {3. + 0. I, 6. + 0. I}}

{{1., 3.}, {-5., 0}, {3., 6.}}


If the ultimate target of this filtering is ListDensityPlot this format will be fine.

I cannot directly compare the performance of getReal on this packed Complex to other methods such as Select[FreeQ[Complex]] and Cases[data, {_Real | _Integer, _Real | _Integer}] as these won't work on the packed data. I shall therefore compare their performance on unpacked big to getReal on the packed form I'll name big2.

big2 = DeveloperToPackedArray[big, Complex];

Select[big, FreeQ[#, Complex] &]                  // Length // RepeatedTiming
Cases[big, {_Real | _Integer, _Real | _Integer}]  // Length // RepeatedTiming
DeveloperToPackedArray[getReal[big2], Real]      // Length // RepeatedTiming

{0.0162, 5000}

{0.01, 5000}

{0.00114, 5000}

{0.00019, 5000}


This doesn't include the initial conversion to packed Complex, which itself takes:

DeveloperToPackedArray[big, Complex]; // RepeatedTiming

{0.00029, Null}


So with that overhead my code is 114 / (19 + 29) ~= 2.3 times faster than Shadowray's next-fastest code, and without that overhead it is almost four times faster.

In:

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}};
data // Select[# ∈ Reals &]


Out:

{{1, 3}, {-5, 0}, {3, 6}}

If you use FreeQ[#,Complex]&, I don't think it's correct. I think the method with the fewest assumptions should be selected. For instance.

In:

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {x, y}}
data // Select[# ∈ Reals &]
Select[data, FreeQ[#, Complex] &]


Out:

{{1, 3}, {-5, 0}, {3, 6}}

{{1, 3}, {-5, 0}, {3, 6}, {x, y}}

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}};
Cases[data, {(_Integer|_Real)..}]
(* {{1, 3}, {-5, 0}, {3, 6}} *)

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {Pi, E}};


Since V 13.3 we have RealValuedNumberQ and RealValuedNumericQ :

Cases[data, {__?RealValuedNumberQ}]


{{1, 3}, {-5, 0}, {3, 6}}

Cases[data, {__?RealValuedNumericQ}]


{{1, 3}, {-5, 0}, {3, 6}, {Pi, E}}

data = {{1, 3}, {-5, 0}, {3, 6}, {2 - I, 1}, {I, -I}, {π, E}}; (* Thanks to  \@eldo *)


Using AllTrue:

Select[AllTrue[Head@# =!= Complex &]][data]


Using Except:

Cases[data, {Except[_Complex] ..}]


Result:

{{1, 3}, {-5, 0}, {3, 6}, {π, E}}