11
$\begingroup$

Is there a way to change the weights of a layer in a neural network, without reconstructing the whole network?

For example,

lenet = NetInitialize@NetChain[{
    ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
    ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
    FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
   "Output" -> NetDecoder[{"Class", Range[0, 9]}],
   "Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
   ]

The weights of the second convolution layer can be extracted using NetExtract

NetExtract[lenet, {4, "Weights"}]

But this is extracted convolution layer is now independent of the neural network. How can I change the weights of this layer in the neural network in place? I'm looking something like

lenet[[4]]["Weights"] = myWeights

Keras has a similar functionality that allows we to modify the weights directly like

layer.set_weights([ker,bia])

Being able to change the weights in place is very useful, for instance, in loading pre-trained weights file.

Improve this question
$\endgroup$
3
  • $\begingroup$ I don't know if there is a way to change the weights of an existing layer, but you can specify the initial weights of a new layer using ConvolutionLayer[... "Weights"->w]. So you could create a new network and initialize all layers to the weights you want $\endgroup$
    – Niki Estner
    Mar 29, 2017 at 19:27
  • $\begingroup$ Also, NetReplacePart looks like a promising way to replace part of an existing network. Haven't tried it, though. $\endgroup$
    – Niki Estner
    Mar 29, 2017 at 19:29
  • $\begingroup$ @nikie NetReplacePart seems to work by return a new network with the updated weights. I somehow missed that function in the documentation. Thanks! $\endgroup$
    – xslittlegrass
    Mar 29, 2017 at 20:17

1 Answer 1

Reset to default
8
$\begingroup$

As nikie suggests, the function NetReplacePart can be used to replace weights of layers in a neural network. It returns a new neural network with the updated weights. For example

myWeights1 = ConstantArray[0., {20, 1, 5, 5}];
myWeights2 = ConstantArray[0., {50, 20, 5, 5}];

lenet = NetReplacePart[lenet, {{1, "Weights"} -> myWeights1, {4, "Weights"} -> myWeights2}]

And we can check that the weights are updated

NetExtract[lenet, {1, "Weights"}] == myWeights1
NetExtract[lenet, {4, "Weights"}] == myWeights2
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.