6
$\begingroup$

Bug introduced in 11.3 or earlier and persisting through 12.0
Bug isn't present in version 8.0.4

I am trying to do clustering and then obtained a plot as below:-

pts = {{1, 1}, {2, 1.5}, {10, 1}, {11, 1.5}};
clts = FindClusters[pts, 2]
ListPlot[clts, PlotRange -> {{0, 12}, {0, 3}}]

enter image description here

As we can see, the 2 points at the left are closer to each other, but they are now put in separated clusters. Why does that happen?

If I reverse the x-y coordinate of the points, the situation is still the same (that means FindClusters is not taking priority for x-coordinates). I then tried FindClusters[pts, 2, CriterionFunction -> "CalinskiHarabasz"], the 2 closer points are still in different clusters. Finally, when I tried FindClusters[pts1, 2, DistanceFunction -> EuclideanDistance], the problem is finally solved and the 2 closer points are in the same cluster.

So I am curious: if the default distance is not the "Euclidean distance", what would that be? Why would it put the closer points into different clusters?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ Bizarre. Trace[FindClusters[pts, 2], HoldPattern[DistanceFunction -> _]] shows a lot of DistanceFunction -> EuclideanDistance, so FindClusters appears to be using EuclideanDistance as default. $\endgroup$
    – JungHwan Min
    Commented Jun 12, 2018 at 21:57
  • $\begingroup$ Works for me: imgur.com/a/RyTK78l $\endgroup$
    – corey979
    Commented Jun 12, 2018 at 22:13
  • $\begingroup$ In this case, FindClusters internally uses MachineLearning`PackageScope`AutomaticDistanceFunction[Automatic][{"Numerical", "Numerical"}] as its distance function, which evaluates to EuclideanDistance. Huh. $\endgroup$
    – JungHwan Min
    Commented Jun 12, 2018 at 22:17
  • $\begingroup$ What OS and version of Mathematica are you using? The code doesn't seem to use EuclideanDistance on Mathematica versions 11.1.1 and 11.2.0 on Windows, and 11.3.0 on Linux. $\endgroup$
    – JungHwan Min
    Commented Jun 12, 2018 at 22:24
  • 2
    $\begingroup$ I observe the same with version 11.3 on Windows 7 x64. Looks like a bug, worth reporting. Version 8.0.4 gives the expected result. $\endgroup$
    – Alexey Popkov
    Commented Jun 13, 2018 at 11:13

1 Answer 1

Reset to default
3
$\begingroup$

When using FindClusters or ClusterClassify there is some preprocessing going on that might change your data. In this case, if you compute 

 c = ClusterClassify[pts, 2]

anche check for c[[1]] you will see that the data gets standardized first. To prevent the function to do that it is enough to specify a DistanceFunction.

When doing it you get the expected answer:

pts = {{1, 1}, {2, 1.5}, {10, 1}, {11, 1.5}};
clts = FindClusters[pts, 2, DistanceFunction -> EuclideanDistance]

{{{1, 1}, {2, 1.5}}, {{10, 1}, {11, 1.5}}}
Improve this answer
$\endgroup$
1
  • $\begingroup$ "there is some preprocessing going on that might change your data" - isn't it a bug? Preprocessing must not change the input data! $\endgroup$
    – Alexey Popkov
    Commented Oct 29, 2019 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.