12
$\begingroup$

Is there a feature to automatically count parameters in a model? In Keras one can use layer.count_params() to get a count of parameters in a layer or model.summary() to get a detailed summary of parameters and layer sizes for the whole net.

$\endgroup$
11
$\begingroup$

As far as I could see I included all layers that have learnable parameters the correct way; if you spot any issues please feel free to leave a comment or correct my answer.

layers with learnable parameters

countparameters[
l_ConvolutionLayer | l_LinearLayer | l_ConstantArrayLayer | 
l_EmbeddingLayer | l_BatchNormalizationLayer | 
l_BasicRecurrentLayer | l_GatedRecurrentLayer | 
l_LongShortTermMemoryLayer] := 
<|NetExtract[l, "Type"] ->
(NetExtract[NetInitialize@l, "Arrays"] //Query[Select[ArrayQ]] //Map[Length@*Flatten])|>

countparameters[l_SpatialTransformationLayer] := 
<|NetExtract[l, "Type"] -> <|"Parameters" -> 6|>|> 

NetChain and NetGraph

countparameters[l_NetChain | l_NetGraph] := l // Normal // Map[countparameters]

other layers

countparameters[_] := Nothing

Example:

lenet = NetChain[{
ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
"Output" -> NetDecoder[{"Class", Range[0, 9]}],
"Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
]

lenet // countparameters
{<|"Convolution" -> <|"Weights" -> 500, "Biases" -> 20|>|>, 
 <|"Convolution" -> <|"Weights" -> 25000, "Biases" -> 50|>|>, 
 <|"Linear" -> <|"Weights" -> 400000, "Biases" -> 500|>|>, 
 <|"Linear" -> <|"Weights" -> 5000, "Biases" -> 10|>|>}

Prototype for a fancy version for use with NetGraph

Take a modular net build with NetGraph such as

module[n_] := 
NetChain[{ConvolutionLayer[n, 5], Ramp, PoolingLayer[2, 2]}]

lenet = NetGraph[
<|
"conv_1" -> module[20],
"conv_2" -> module[50], 
"flatten" -> FlattenLayer[],
"dense" -> NetChain[{LinearLayer[500], Ramp, LinearLayer[10], SoftmaxLayer[]}]
|>, 
{"conv_1" -> "conv_2" -> "flatten" -> "dense"}, 
"Output" -> NetDecoder[{"Class", Range[0, 9]}], 
"Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]]

then something like

summary[net_NetGraph] := Module[{sum, parameterCounts},
sum[x_Integer] := x;
sum[x : <|(_ -> _Integer) ..|>] := Total@Values@x;
sum[x_List] := x // Total;
sum[_] := 0;

parameterCounts = net // countparameters;

<|"Total" -> (parameterCounts // Map[sum, #, {0, Infinity}] &) , 
"Details" -> (parameterCounts // Map[sum, #, {1, Infinity}] &) |>
]

gives a nice overview how the learnable parameters are distributed throughout the network architecture.

lenet // summary
<|
 "Total" -> 431080, 
 "Details" ->
 <|"conv_1" -> 520, "conv_2" -> 25050, "flatten" -> 0, "dense" -> 405510|>
|>
$\endgroup$
  • $\begingroup$ I think in the spirit of your question it is better to have GroupBy[#, First@*Keys -> Values, Total]& at the end of countparameters[l_NetChain | l_NetGraph] . I.e. to have a definition like this: GroupBy[l // Normal // Map[countparameters], (First@*Keys) -> Values, Total] . $\endgroup$ – Anton Antonov Jun 30 '17 at 11:07
  • $\begingroup$ @AntonAntonov See the edit at the end of my post. $\endgroup$ – Sascha Jun 30 '17 at 11:13
  • $\begingroup$ I added correct handling of layers with setting "Biases" -> None $\endgroup$ – Sascha Jul 10 '17 at 9:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.