# How to change weights of a neural network

Is there a way to change the weights of a layer in a neural network, without reconstructing the whole network?

For example,

lenet = NetInitialize@NetChain[{
ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
"Output" -> NetDecoder[{"Class", Range[0, 9]}],
"Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
]


The weights of the second convolution layer can be extracted using NetExtract

NetExtract[lenet, {4, "Weights"}]


But this is extracted convolution layer is now independent of the neural network. How can I change the weights of this layer in the neural network in place? I'm looking something like

lenet[[4]]["Weights"] = myWeights


Keras has a similar functionality that allows we to modify the weights directly like

layer.set_weights([ker,bia])


Being able to change the weights in place is very useful, for instance, in loading pre-trained weights file.

• I don't know if there is a way to change the weights of an existing layer, but you can specify the initial weights of a new layer using ConvolutionLayer[... "Weights"->w]. So you could create a new network and initialize all layers to the weights you want – Niki Estner Mar 29 '17 at 19:27
• Also, NetReplacePart looks like a promising way to replace part of an existing network. Haven't tried it, though. – Niki Estner Mar 29 '17 at 19:29
• @nikie NetReplacePart seems to work by return a new network with the updated weights. I somehow missed that function in the documentation. Thanks! – xslittlegrass Mar 29 '17 at 20:17

## 1 Answer

As nikie suggests, the function NetReplacePart can be used to replace weights of layers in a neural network. It returns a new neural network with the updated weights. For example

myWeights1 = ConstantArray[0., {20, 1, 5, 5}];
myWeights2 = ConstantArray[0., {50, 20, 5, 5}];

lenet = NetReplacePart[lenet, {{1, "Weights"} -> myWeights1, {4, "Weights"} -> myWeights2}]


And we can check that the weights are updated

NetExtract[lenet, {1, "Weights"}] == myWeights1
NetExtract[lenet, {4, "Weights"}] == myWeights2