# Learning weights during training

Let's assume I have a net so the loss function is composed from 3 components: $$C_1, C_2, C_3$$, and each component is multiply by its weight $$(w1, w2, w3)$$. The weights should be a function of $$\beta$$, which is a function of the error/error rate/std etc, so the loss function can be written:

$$L=C_1 w_1(\beta) + C_2 w_2(\beta) + C_3 w_3(\beta)$$

In Mathematica, the loss would be:

  L = w1[beta]*C1 + w2[beta]*C2 + w3[beta]*C3


The net I have is shown in the diagram below.

The weights should be normalized I am guessing, and sum to one.

I would like the network to be able to learn the optimal weights. How can I do it and which layers I need to add to my network? How can I use the error and calculate the error rate or the standard deviation of the error signal, during the training process ?

Edit: The code that produced the above network is:

lossNet2 = NetGraph[
<|"Net" -> UNET,
"C1" -> lossByWeights,
"C2" -> lossByDev,
"C3" -> lossByMinMax
|>,
{
NetPort[ "Input"] -> "Net", "Net" -> NetPort["Output"],
{"Net", NetPort[ "Target"]} -> "C1" -> "finalAddition",
"Net" -> "C2" -> "finalAddition" -> NetPort["Loss"],
{NetPort[ "Input"], NetPort["Target"]} -> "C3" -> "finalAddition"
},
"Input" -> {1, 1, 512}]


Any help will be appreciate.

Thanks,

Guy

• When editing your question, I thought your objective function was the sum of the terms. Is this correct? Or do you have a multi-objective function (which would be a more complex problem. Thanks – mjw Apr 3 at 13:19
• I think it would be helpful to include the code you used to produce the network. – mjw Apr 3 at 13:22
• I added the code, thanks – GM_jnj Apr 4 at 5:17
• Getting this error message: "NetGraph::netinvnodes: UNET is not a net function." What is UNET? – mjw Apr 4 at 14:20
• UNET is a customized net that I built. The specific net is not relevant, It can be any net. – GM_jnj Apr 7 at 4:33