I basically retrieved the following technique of evaluating the mutual information involving two matrices from this site at http://bmia.bmt.tue.nl/People/BRomeny/Courses/8C080/default.htm The original code was written for image-processing, but could perhaps apply to other areas as well.
Mutual Information = H(A)+H(B)-H(AB), where H(X) is the entropy of X
My question: How to do you generalize this to apply to a matrix of arbitrary elements (including decimals less than 1, and negative numbers)? Can this also be extended to matrix of any dimension (i.e. larger than 2?).
The program follows as:
imA = ({
{0, 1, 3, 0},
{0, 0, 2, 5},
{3, 2, 0, 1},
{0, 0, 3, 4}
});
imB = ({
{1, 1, 0, 4},
{3, 0, 4, 5},
{0, 0, 2, 5},
{0, 1, 4, 5}
});
histogramA = BinCounts[Flatten[imA], {0, 6, 1}]
hA = Total[-(histogramA/nA) Log[histogramA/nA]] // N
nA = Total[histogramA]
histogramB = BinCounts[Flatten[imB], {0, 6, 1}];
nB = Total[histogramB];
hB = Total[-(histogramB/nB) Log[histogramB/nB]] // N
imAB = Transpose[{imA, imB}, {3, 1, 2}]
histogramAB = BinCounts[Flatten[imAB, 1], {0, 6, 1}, {0, 6, 1}];
nAB = Total[histogramAB, 2];
fh = Flatten[histogramAB]
fh2 = DeleteCases[fh, 0]
hAB = Total[-(fh2/nAB) Log[fh2/nAB]] // N
MI = hA + hB - hAB
a
by:Round[255*a/(Max[Max[a]] - Min[Min[a]])]
$\endgroup$ – bill s Jul 16 '13 at 0:52Library
NConditionalEntropy[imA, imB] is the required answer. $\endgroup$ – thils Jul 16 '13 at 12:44