A few weeks ago I ran a cluster analysis on some economic data and (unfortunately only) now I found out that some results do not make sense. For instance I analyzed the development of unemployment rates form end-2005 to mid-2011 for 32 countries. One cluster contained among others Germany (Figure 1) and Spain (Figure 2) and another cluster among others Estonia (Figure 3). I couldn't understand why Mathematica would put Germany and Spain together in one cluster, instead of Estonia and Spain. I just calculated the EuclideanDistance
between Germany and Spain (17.3) as well as Spain and Estonia (9.1). These results underline my doubts on my cluster-analysis-results.
Here is the code with which I worked:
First I generate a random sample (n=1200) of the original data, to minimize the effect of the input-order in FindClusters
(Note from Mathematica-Documentation: "The order of elements can have an effect on the clusters found")
data; (*see below*)
Table[DeleteDuplicates[
Table[RandomSample[DeleteCases[data[[j]], {}]], {i, 1200}]], {j,Length[data]}];
Then I run FindCluster
for the first time to determine the commonest length of each cluster.
ClustersUnSorted[list_]:=Table[Map[FindClusters,list[[i]]],{i, Length[list]}]
In my next step I say, that if the previous step found less than four clusters, I would like Mathematica instead to set the minimum number of clusters to 4.
ClusterLength[list_]:=Table[If[
Commonest[Map[Length,list[[i]]]][[1]]<4,
4, Commonest[Map[Length, list[[i]]]][[1]]],
{i, Length[list]}]
Then I run FindClusters
again on the basis of the previously determined number of clusters.
ClustersUnSortedFixLength[list_,Flatten[ClusterLength_]]:=
Table[Map[FindClusters[#,ClusterLength[[i]]] &, list[[i]]], {i, Length[list]}]
The last two steps are there to get rid of duplicates as for each indicator I generated a random sample (see step 1).
ClustersSorted[list_] := Table[Table[
Sort[Map[Sort, list[[j]][[i]]]], {i, Length[list[[j]]]}],
{j, Length[list]}]
CommonestCluster[list_] := Map[Commonest, list]
The last two steps can be easily comprehended by the following example (instead of RandomSample
I could use Permutations[{1, 5, 3, 10, 100}]
for a much more reliable result but since Permutations
is limited to input with length less than 11 it is not applicable to my data.):
Table[RandomSample[{1, 5, 3, 10, 100}], {i, 1200}];
Map[FindClusters, %];
Table[Map[Sort, %[[i]]], {i, Length[%]}];
Map[Sort, %];
Commonest[%]
exemplary data (Note: the time series are not always of the same length.):
Country1GDP = Join[{{"Country1", "GDP"}}, RandomReal[{-1, 1}, {6}]];
Country2GDP = Join[{{"Country2", "GDP"}}, RandomReal[{-1, 1}, {6}]];
Country3GDP = Join[{{"Country3", "GDP"}}, RandomReal[{-1, 1}, {6}]];
Country4GDP = Join[{{"Country4", "GDP"}}, RandomReal[{-1, 1}, {5}]];
Country5GDP = Join[{{"Country5", "GDP"}}, RandomReal[{-1, 1}, {6}]];
Country1Imports =
Join[{{"Country1", "Imports"}}, RandomReal[{-1, 1}, {6}]];
Country2Imports =
Join[{{"Country2", "Imports"}}, RandomReal[{-1, 1}, {5}]];
Country3Imports =
Join[{{"Country3", "Imports"}}, RandomReal[{-1, 1}, {6}]];
Country4Imports =
Join[{{"Country4", "Imports"}}, RandomReal[{-1, 1}, {6}]];
Country5Imports =
Join[{{"Country5", "Imports"}}, RandomReal[{-1, 1}, {6}]];
CompleteGDP = {Country1GDP, Country2GDP, Country3GDP, Country4GDP,
Country5GDP};
CompleteImports = {Country1Imports, Country2Imports, Country3Imports,
Country4Imports, Country5Imports};
data = {CompleteGDP, CompleteImports};
Table
call... i.e., you can writeTable[..., {i,...}, {j,...}]
instead ofTable[Table[...,{i,...}], {j,...}]
$\endgroup$s
instead ofz
, which I think we should respect... $\endgroup$data
to start out with? (Sorry if I missed it) $\endgroup$