7
$\begingroup$

I have to find the probability density function of the output of an internal layer in a neural network.

I wanted to do it using NetMeasurements adding a layer like

ElementwiseLayer[BinCounts[{#}, {0, 1, .05}] &]

to my ElementwiseLayer[LogisticSigmoid] and measuring its average over the dataset. However, I get the error that it "could not be symbolically evaluated as a unary scalar function."

Therefore I have to apply NetMeasurements to each element of the dataset to get the output of my layer and then calculate BinCounts, which is very slow.

Is there a way to calculate PDFs of the layer's output faster?

$\endgroup$

1 Answer 1

3
$\begingroup$

Here is how one can implement BinCounts as a layer:

ClearAll[binCountsLayer];

binCountsLayer[{start_, end_, step_}] :=
 NetChain[{CompiledLayer[
    Function[{Typed[input, 
       TypeSpecifier["NumericArray"]["Real64", 1]]}, 
     Quotient[Clip[input, {start - step, end}] - start, step] + 2], None],
   UnitVectorLayer[Quotient[end - start, step] + 2], 
   AggregationLayer[Total, 1], PartLayer[2 ;; -2]}]

Test

Block[{temp = RandomReal[{-10, 20}, 50]},
 {binCountsLayer[{0., 10., 1.}][temp], BinCounts[temp, {0, 10, 1.}]}
 ]

(* Out: {{0, 0, 1, 0, 1, 2, 2, 4, 1, 1},
         {0, 0, 1, 0, 1, 2, 2, 4, 1, 1}} *)
Block[{temp = RandomReal[{-1, 2}, 50]},
 {binCountsLayer[{0., 1., .25}][temp], BinCounts[temp, {0, 1, .25}]}
 ]

(* Out: {{5, 5, 6, 3},
         {5, 5, 6, 3}} *)

Note that the input with start/end/step arguments should be the same type specified in the CompiledLayer (Real64).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.