0
$\begingroup$

Limit us with Clustering technology, like FindClusters, ClusteringComponents,ClusterClassify,FeatureExtraction,DimensionReducion.

resource = ResourceObject["MNIST"];
trainingData=ResourceData[resource,"TrainingData"];
testData=ResourceData[resource,"TestData"];
trainingSet=RandomSample[trainingData,5000];
testSet=RandomSample[testData,1000];
methods = {"KMeans", "Agglomerate", "KMedoids", "Spectral"};
Do[clusters[i]=FindClusters[dTrain,10,Method->i,DistanceFunction->CosineDistance,PerformanceGoal->"Quality"];
length=Length/@clusters[i];
Print@length,{i,methods}]

---Basic effect

fe=DimensionReduction[trainingSet[[All,1]],Method->"TSNE"];
dTrainDR=Thread[fe[trainingSet[[All,1]]]->trainingSet[[All,2]]];
Do[c[i]=ClusterClassify[dTrainDR[[All,1]],10,Method->i,PerformanceGoal->"Quality"],{i,methods}]
asso=GroupBy[dTrainDR,Last->First];
KeySort[ReverseSort@Counts[c[methods[[1]]]@#]&/@asso]//Normal//Column
0-><|2->446,10->15,4->3,7->1|>
1-><|6->422,8->130,1->12,5->3,4->1|>
2-><|4->408,6->69,8->16,1->8,2->8,5->6,10->5,9->2,3->1|>
3-><|7->405,8->32,5->23,1->5,10->5,2->4,4->3,6->3,9->2,3->1|>
4-><|3->224,1->153,5->72,9->47,6->14,10->2,8->1|>
5-><|7->192,8->149,5->108,10->22,3->6,2->3,1->2,6->2|>
6-><|10->483,8->3,4->2,5->2,3->1,6->1,2->1,7->1|>
7-><|9->259,1->178,5->37,6->17,3->15,4->2,10->1|>
8-><|8->222,5->138,7->66,10->12,1->10,6->8,3->6,2->5,4->1|>
9-><|3->162,1->146,9->124,5->51,7->4,6->2,8->2,2->2|>

and we see, 4 and 9 is similar, 5 and 3 is similar, the question is not limit to Hard Clustering, also Soft Clustering, we can raise up some evaluation, topK accuracy like some evaluating indicators in Classify.

I'll update my experiments and notebook here or some places, glad to know more about this topic.my experiment notebook

For example, more accuracy, what feature, what distance, what cluster method, what dimension reduce function gives better result?

$\endgroup$
  • 1
    $\begingroup$ I don't follow. What exactly is your question? $\endgroup$ – MarcoB Aug 31 '17 at 13:00
  • $\begingroup$ @MarcoB For example, what feature, what distance, what cluster method, what dimension reduce function gives better result? $\endgroup$ – HyperGroups Aug 31 '17 at 13:14
  • $\begingroup$ Maybe TSNE will help. $\endgroup$ – partida Sep 11 '17 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.