# Machine learning: How to fix part of the weight matrix?

Consider a linear layer with 2*2 weights

linear = LinearLayer[2, "Input" -> 2];

net = NetInitialize[linear];


After the initialization, the four elements of the weight matrix W can be read

W = NetExtract[net, "Weights"]
(* {{0.551202, -1.10384}, {-0.659195, -1.12961}} *)

data = {{1, 1} -> {2, 3}, {2, 2} -> {4, 5.9}, {3, 3} -> {6, 9.1}, {4,
4} -> {8, 12.1}};

trainednet = NetTrain[net, data];

trainednet[{7, 7}]
(* {14., 21.25} *)


After training, the initial four weights W have been changed

NetExtract[trainednet, "Weights"]
(* {{1.82752, 0.17248}, {1.76021, 1.28979}} *)


Question: Is it possible to fix three of the weights (e.g., W[[1,1]]=1,W[[1,2]]=1,W[[2,1]]=1 ) so that only one weight (W[[2,2]]) can be changed during the training?

• It will help if you provide additional input. May be a MWE, references, etc.... – Nasser Dec 5 '18 at 16:56
• Thank Jason and Nasser. I have re-edited the question. – chrestun Dec 6 '18 at 8:21
• I would restate the question as an ability to specify a hook for optimization step, in order to mask the gradient however I want. – swish Dec 6 '18 at 19:02

net = NetGraph[
{
ConstantArrayLayer[{3}, "Array" -> {1, 1, 1}],
ConstantArrayLayer[{1}],
CatenateLayer[],
ReshapeLayer[{2, 2}],
DotLayer[],
2,
SoftmaxLayer[]
},
{
NetPort["Input"] -> 5, {1, 2} -> 3 -> 4 -> 5 -> 6 -> 7
},
"Input" -> 2,
"Output" -> NetDecoder[{"Class", {0, 1}}]
] // NetInitialize


NetExtract[net, {1, "Array"}]


{1., 1., 1.}

NetExtract[net, {2, "Array"}]


{0.}

netT = NetTrain[
net,
RandomReal[{-1, 1}, {100, 2}] -> RandomInteger[{0, 1}, 100],
LearningRateMultipliers -> {1 -> 0, _ -> 1},
MaxTrainingRounds -> 10
]


NetExtract[netT, {1, "Array"}]


{1., 1., 1.}

NetExtract[netT, {2, "Array"}]


{-0.0881761}

• It really is what I want. Thank you so much. – chrestun Dec 9 '18 at 14:58