Suppose I have a list of groups:
{{1,2,3,4}, {1,2}, {3,4}}
In this example, 1
most commonly appears within a group that contains 2
, and 3
most commonly appears in a group which contains 4
. If we form N
subgroups, where N==2, the best-fitting subgroups by frequency of grouping would be {{1,2}, {3,4}}
.
As a second example, a list of groups could be defined as:
{{1,2,3}, {1,2}, {2,3}, {3,4}}
In this example:
1
is in a group with2
: 2/2 times1
is in a group with3
: 1/2 times1
is in a group with4
: 0/2 times2
is in a group with1
: 2/3 times2
is in a group with3
: 2/3 times2
is in a group with4
: 0/3 times3
is in a group with1
: 1/3 times3
is in a group with2
: 2/3 times3
is in a group with4
: 1/3 times4
is in a group with1
: 0/1 time4
is in a group with2
: 0/1 time4
is in a group with3
: 1/1 time
Such that a valid subset grouping would include {{1,2,3},{4}}
but not {1,2,3,4}
(since 1
is never grouped with 4
). I'm not quite sure how one would score the alternative groupings to rank {{1,2,3},{4}}
against another possible grouping like {{1,2}, {3,4}}
to determine the best-fitting options.
I'm open to the idea of allowing multiple subgroups to include the same item, but the number of groups returned should be manageable for large collections of unique items, such as not to explode into a full set of combinations.
With a large collection of lists, how might I divide the unique items across all sets into the best fitting subgroups, defined by the most common frequencies of the groupings?