# FindClusters for 3D Pancake/Platelet Shapes

I'm looking to properly cluster a dataset of roughly 3 million data points in 3-space. The shapes form closely spaced "clusters" that resemble pancakes.

Here is a downsampled dataset:

data = Import["https://dl.dropbox.com/u/28603777/data.wdx"];


I have tried the standard distance methods (Canberra, Euclidean, Manhattan and BrayCurtis), yet each leaves some points from one cluster improperly sorted into another.

out = FindClusters[data, 6];


The navy platelet "leaks" into the gold platelet due to the fact that the distance methods are isotropic. Also the gold platelet "leaks" into the red platelet.

Is there a different way to cluster such that the distance method can be optimized in this way, or is there a fast way to cluster these platelets that does not rely on this function.

-
Not sure if I properly understand your question, but adding Method -> "Agglomerate" to the FindClusters function seems to do the trick - not sure if that helps for the whole set of data... –  Pinguin Dirk Mar 29 '13 at 17:20
@PinguinDirk Could make that an answer –  Sjoerd C. de Vries Mar 29 '13 at 18:45

Based on the data you provide, it seems that hierarchical clustering (see wiki here) with type "agglomerate" (bottom up) solves your problem, i.e.:

out = FindClusters[data, 6, Method -> "Agglomerate"];
ListPointPlot3D[out]


and get:

Based on how your full dataset looks like (e.g. if you know how many clusters there are etc.), you might need to adapt the code a bit (possibly also with distance function or matrix) - but for the sample you are providing, it seems to work nicely. Also, performance could be an issue, as noted in the wiki article.

-