10
$\begingroup$

Here is my simple example, and in this case function DeleteDuplicates does not work as expected.

I want to FindRoot of my function $\chi[\nu]$, and since function $\chi$ is very sensitive to initial guess I decided to generate many initial conditions and leave only those solutions which are distinct. For this purpose I want to apply DeleteDuplicates on the resulting list of solution.

Here is the definition of my function:

χ[ν_] := 2*PolyGamma[1] - PolyGamma[1/2 + I*ν] - PolyGamma[1/2 - I*ν]

Here I generate many solution according to many random initial guesses

m = Table[v /. FindRoot[χ[v] == -1.2 - 0.2*I, {v, RandomComplex[]}], {i, 1, 10}]

And finally, I want to leave only distinct solutions:

DeleteDuplicates[m]

Unfortunately, the operation DeleteDuplicates[m] does not change the list m, although there are many identical values.

Namely:

DeleteDuplicates[m]

{1.06423 + 0.0968739 I, 1.06423 + 0.0968739 I, 
 1.06423 + 0.0968739 I, 1.06423 + 0.0968739 I, 0.0250407 + 1.00352 I, 
 1.06423 + 0.0968739 I, 1.06423 + 0.0968739 I, 1.06423 + 0.0968739 I, 
 0.0250407 + 1.00352 I, 1.06423 + 0.0968739 I}

I'm puzzled.

Any help or suggestions are very welcome!

Thanks!

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Try something like DeleteDuplicates[m, Abs[#1 - #2] < 0.01 &] . $\endgroup$
    – b.gates.you.know.what
    Mar 20, 2013 at 9:59
  • $\begingroup$ Thanks for editing Mr. Wizard! It looks much better :) @b.gatessucks: Great, works like a charm. Could you explain the reason? I'm quite new to this. Thanks a lot for help! $\endgroup$
    – Arnold Klein
    Mar 20, 2013 at 10:03
  • $\begingroup$ Andrew, please see the update to my answer for an important note about performance. $\endgroup$
    – Mr.Wizard
    Mar 21, 2013 at 13:53
  • 1
    $\begingroup$ Andrew, what about changing "does not work properly" to "does not work as expected"? In fact it works properly, the cause is just seemingly identical numbers... $\endgroup$
    – Yves Klett
    Mar 21, 2013 at 14:17
  • $\begingroup$ Related: (19112) $\endgroup$
    – Mr.Wizard
    Feb 19, 2016 at 21:50

2 Answers 2

Reset to default
12
$\begingroup$

You would do well to understand the difference between tools that are intended for structural operations and those that are intended for mathematical operations. DeleteDuplicates is of the former, generally speaking. As such it is comparing the exact FullForm of the objects, or at least something close (caveat).

As b.gatessucks recommends in a comment you can use a mathematical comparison function for the equivalence test of DeleteDuplicates, e.g.:

 DeleteDuplicates[m, Abs[#1 - #2] < 10^-12 &]

{1.06423 + 0.0968739 I, 0.0250407 + 1.00352 I}

Incidentally you could also use Union, but the syntax is a bit different. Note the ( ).

 Union[m, SameTest -> (Abs[#1 - #2] < 10^-12 &)]

{0.0250407 + 1.00352 I, 1.06423 + 0.0968739 I}

Using InputForm to show all of the digits of your expression you can see that they are not structurally identical in the (approximate) way that Mathematica "sees" them:

m // InputForm

{1.0642275928442373 + 0.09687392021742822*I, 1.0642275928442366 + 0.09687392021742817*I, 
 1.0642275928442366 + 0.09687392021742797*I, 1.064227592844237 + 0.09687392021742822*I, 
 1.0642275928442373 + 0.09687392021742852*I, 1.0642275928442366 + 0.09687392021742793*I, 
 1.0642275928442368 + 0.09687392021742801*I, 0.025040728196256346 + 1.0035162552538588*I, 
 1.0642275928442377 + 0.0968739202174282*I, 1.0642275928442375 + 0.0968739202174283*I}

Performance

Yves reminded me to mention something about the performance of using a custom comparison function in DeleteDuplicates or Union as I did above. For long lists this is always considerably slower than using the default method. I gave an example with timings in How to represent a list as a cycle.

To apply that method here we could Round the numbers beforehand:

Round[m, 10^-12] // DeleteDuplicates // N

{1.06423 + 0.0968739 I, 0.0250407 + 1.00352 I}

I added // N to convert back to machine precision, but the values will not be precisely the same. This probably doesn't matter if you consider numbers this close to be duplicates, but should you want the unchanged numbers you could use GatherBy and get performance not far distant.

First /@ GatherBy[m, Round[#, 10^-6] &]

Version 10.0 introduced DeleteDuplicatesBy which works similarly to the GatherBy method; it has the following syntax:

DeleteDuplicatesBy[m, Round[#, 10^-6] &]

However it may not perform as well as GatherBy; see:

Improve this answer
$\endgroup$
12
  • $\begingroup$ Thanks for the answer, I see. It's like a comparison with epsilon of two floating point numbers. I assumed, that MATHEMATICA already does it for me. $\endgroup$
    – Arnold Klein
    Mar 20, 2013 at 10:08
  • $\begingroup$ Great, I missed InputForm. Without it it looks the same. $\endgroup$
    – Arnold Klein
    Mar 20, 2013 at 10:09
  • 1
    $\begingroup$ @Andrew FullForm was a bit needlessly verbose here, but in general you should look at the FullForm when trying to understand how Mathematica will treat an expression using structural tools. There are many cases where not using it will leave you quite befuddled as it can be very different from what is shown in standard output notation. $\endgroup$
    – Mr.Wizard
    Mar 20, 2013 at 10:13
  • 1
    $\begingroup$ Admonish - lovely acoustics but far too harsh. I thought of it as "gently encouraging" at most... $\endgroup$
    – Yves Klett
    Mar 21, 2013 at 14:13
  • 1
    $\begingroup$ @AJHC Converting to plain lists using {x, y, z} /. solutions seems like a good way to start. For example with m2 = {{x -> 0.1, y -> 0.2, z -> 0.3}, {x -> 0.1, y -> 0.2, z -> 0.3}, {x -> 0.17, y -> 0.22, z -> 0.314}} then DeleteDuplicates[{x, y, z} /. m2, AllTrue[Abs[# - #2], # < 10^-5 &] &] or DeleteDuplicatesBy[{x, y, z} /. m2, Round[#, 10^-5] &] for the two fundamentally different methods described in my answer. If you need to convert back to a list of rules then something like Thread[{x, y, z} -> #] & /@ {{0.1, 0.2, 0.3}, {0.17, 0.22, 0.314}} $\endgroup$
    – Mr.Wizard
    Mar 25, 2019 at 12:05
1
$\begingroup$

Note also that one can use Equal[##] & but not Equal:

DeleteDuplicates[m, Equal[##] &]
(*  {1.06423 + 0.0968739 I, 0.0250407 + 1.00352 I}  *)

DeleteDuplicates[m, Equal] // Length
(*  10  *)

This should work the same on most or all well-behaved FindRoot results.

As already noted, using something like Equal[##] & causes the performance to degrade, significantly on very long lists, but using Equal does not cause the same problem. However, Equal does not tolerate difference, unlike Equal[##] &:

DeleteDuplicates[{1., 1. + $MachineEpsilon}, Equal]
    DeleteDuplicates[{1., 1. + 64 $MachineEpsilon}, Equal[##] &]
(*
  {1., 1.}
  {1.}
*)

Basically DeleteDuplicates[.., Equal] is the same as DeleteDuplicates[].

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.