One way to define perm
would be to consider all permutations of the rows and columns of m
, then take the resulting matrix elements to be base-2 digits of an integer number uniquely identifying each permuted matrix. The matrix presenting the minimal such number among all 8! permutations can then become your canonical matrix, i.e.
perm[m_] := First@MinimalBy[Table[m[[p, p]], {p, Permutations@Range@8}], FromDigits[Flatten@#, 2] &]
Of course, this is quite slow since each call to perm results in ~40.000 operations, but the problem you are trying to solve is in fact hard, in the sense that it can be reinterpreted as addressing the graph isomorphism problem: taking m
to be the adjacency matrix of a graph, finding a canonical form for m
is equivalent to relabeling the nodes in a canonical way, which could be used to check whether two graphs are isomorphic or not... and we know this to be an NP-complete problem. [EDIT: We don't know this, see comments below]
The observation above, however, leads to a better implementation of perm
, taking advantage of Mathematica's pre-implemented heuristics for graph canonization:
perm[m_] := AdjacencyMatrix@CanonicalGraph@AdjacencyGraph@m
This should be faster than the "naive" implementation, but still too slow for large matrices (or lots of calls)...