2
$\begingroup$

Obtain a trained network from NetTrain as

net = NetTrain[net, dat];

which i believe returns a ClassifierFunction (although Head[net] returns NetChain).

In the example docs for NetTrain it says:

From a random sample, select the images for which the net produces highest and lowest entropy predictions. High entropy inputs can be interpreted as those for which the net is most uncertain about the correct class:

images = RandomSample[Keys[dat], 5000];
entropies = net[images, "Entropy"];

My question is: what is "Entropy" here and how is it calculated? Is the entropy returned similar to entropy from ImageMeasurments or is it actually based on some information produced by the network etc?

$\endgroup$
1
$\begingroup$

Look at the source code:

C:\Program Files\Wolfram Research\Mathematica\11.0\SystemFiles\Components\NeuralNetworks\Decoders\Class.m

DEntropy = Compile[{{probs, _Real, 1}}, Module[{e = 0.},
    Do[
        If[2*$MachineEpsilon <= p <= 1., e -= p * Log[p]],
        {p, probs}
    ];
    e
], RuntimeAttributes -> {Listable}];
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.