Pairwise similarity comparison

Question

I have a 444-member array of arrays called final, with each subarray containing 12 integer members. How can I do a pairwise comparison of the contents of the subarrays to find all pairs with 11 members in common ; failing that, all those with 10 in common ; etc. ?

Is there a built-in similarity measure for numeric arrays?

EDIT for clarity : I would be happy if the function returned the pair with maximum similarity, or an array of such pairs in the case of a multi-way tie.

Seeking the function Foo

In[1]:= bar = RandomInteger[9, {6, 4}]
Out[1]= {{6, 6, 1, 5}, {4, 0, 9, 3}, {0, 4, 2, 9},
          {4, 6, 0, 6}, {2, 5, 4, 8}, {4, 1, 9, 1}}

In[2]:= Foo[%]
Out[2]= {{4, 0, 9, 3}, {0, 4, 2, 9}}

In[3]:= Foo[bar]
Out[3]= {bar[1], bar[2]}

Answer 1

Update: a single function that combines all the steps:

ClearAll[sF, sortedF];
sF = -Length[Intersection @@ #] &;
sortedF = With[{s = Subsets[#, {2}]}, SortBy[s, sF]] &;

First@sortedF@bar
(* {{4, 0, 9, 3}, {0, 4, 2, 9}} *)

sortedpairs = sortedF@bar;
Grid[{#, -sF@#} & /@ sortedpairs]

---

Original post

sF = -Length[Intersection @@ #] &;

Examples:

OP's example:

bar= {{6, 6, 1, 5}, {4, 0, 9, 3}, {0, 4, 2, 9}, {4, 6, 0, 6}, {2, 5, 4, 8}, {4, 1, 9, 1}};
SortBy[Subsets[bar, {2}], sF][[1]]
(* {{4, 0, 9, 3}, {0, 4, 2, 9}} *)

All pairs sorted by the cardinality of intersection:

SeedRandom[123];
arrays = RandomInteger[9, {5, 4}];
pairs = Subsets[arrays, {2}];
srtdpairs = SortBy[pairs, sF];
Grid[Thread[{{"arrays", "pairs", "sorted\npairs"}, Join @@ {{Column@arrays}, 
      Grid[Join[{{"pair", "similarity"}}, {#, -sF@#} & /@ #]] & /@ {pairs, srtdpairs}}}],
 Alignment -> Center, Dividers -> All]

Update: A MatrixPlot of the pairwise similarities for a 50 x 12 array:

SeedRandom[12];
ClearAll[sF, sortedF];
sF = -Length[Intersection @@ #] &;
arrays = RandomInteger[9, {50, 12}];
MatrixPlot[Outer[-sF[{##}]/9 &, arrays, arrays, 1], ImageSize -> 500,
           ColorFunction -> "SolarColors", ColorFunctionScaling -> False]

Answer 2

EDIT

There were some issues with my original post that I have modified.

In case you wish to count {6,6,3},{6,6,2} as having 2 elements in common rather than 1 (as Intersection produces), I present this rather ugly code. Note assumes integers from 0 to 9. Could be adapted.

func[data_, a_, n_] := 
 Module[{var = Table[Unique["x"], {Length[a]}], r, rl, sub, f, pck, 
   res},
  r = Thread[a -> var];
  rl = Thread[var -> E];
  sub = Subsets[data, {2}];
  f = Times @@ (# /. r) &;
  pck = (Log[#] /. rl) == n & /@ PolynomialGCD @@@ Map[f, sub, {2}];
  res = Normal[GroupBy[Pick[sub, pck], First -> Last]];
  Join @@ Function[{x, y}, {x, #} & /@ y] @@@ res
  ]

where a is alphabet and n is then number of elements that agree. The output is now more like kguler but with counting repeats:

Column[# -> 
    Column[Row /@ 
      func[{{6, 6, 1, 5}, {4, 0, 9, 3}, {0, 4, 2, 9}, {4, 6, 0, 
         6}, {2, 5, 4, 8}, {4, 1, 9, 1}}, Range[0, 9], #]] & /@ 
  Range[0, 4], Frame -> All]

More cumbersome example: r

i = {{0, 6, 3, 8}, {8, 8, 8, 0}, {1, 1, 6, 7}, {7, 0, 2, 1}, {5, 1, 
   3, 5}, {5, 9, 7, 9}, {2, 9, 8, 9}, {5, 4, 5, 2}, {5, 8, 5, 8}, {6, 
   4, 0, 6}, {6, 3, 3, 1}, {1, 1, 1, 0}, {1, 0, 5, 2}, {2, 3, 1, 
   0}, {2, 2, 7, 7}, {6, 1, 2, 1}, {0, 9, 3, 6}, {2, 9, 2, 3}, {0, 8, 
   2, 7}, {3, 7, 8, 0}}
FlipView[Framed /@ (# -> 
      Row[{Column[Row /@ func[ri, Range[0, 9], #]], 
        Length@(func[ri, Range[0, 9], #])}] & /@ Range[0, 4])]