9
$\begingroup$

I want to make a DNN that it can learn from text feature(1534-dims) and old audio features(256*4(four old frames)-dims) to predict audio feature(256-dims) at current frame.

So it has two inputs,one is text feature(1534-dims),the other is old audio features(1024 dims).And one output is current audio feature(256-dims).

First I want net can learn how to reduce dimensions from 1534-dims to 128-dims,so I define encodeNet and decodeNet.

encodeNet = NetChain[{512, Ramp, 128}, "Input" -> 1534(*text feature*)];
decodeNet = NetChain[{512, Ramp, 512, Ramp, 256}, 
                     "Input" -> 128(*low rank representation of text feature*)];

Second I want to combine this reduced dimensions 128-dims and old audio features 1024-dims,then use it to predict the current audio features.

net = NetGraph[{encodeNet, decodeNet, CatenateLayer[], 512, Ramp, 512,Ramp, 256}, 
           {NetPort["Input1"] -> 1 -> 2 -> NetPort["Output1"], 
           NetPort["Input2"] -> 3, 1 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> NetPort["Output2"]}, 
           "Input1" -> 1534, "Input2" -> 256*4, "Output1" -> 256, "Output2" -> 256]

enter image description here

So the data into output1 is same as the data into output2.

input1 = N@RandomInteger[1, {100, 1534}];
input2 =   RandomReal[1, {100, 256*4}];
output =   RandomReal[1, {100, 256}];

data = <|"Input1" -> input1, "Input2" -> input2, "Output1" -> output,"Output2" -> output|>;

Then train the net.

net = NetTrain[net, data, MaxTrainingRounds -> 100];

enter image description here

So how does mathematica compute loss with multi-losses? is it loss=loss1+loss2?

And is there better net than can deal with this problem? I want to use EmbeddingLayer to replace encodeNet, but have no idea.

And can it have one ouput port and how to use it?

$\endgroup$
  • $\begingroup$ There's no support for multiple losses yet $\endgroup$ – M.R. May 22 '17 at 1:24
  • $\begingroup$ @M.R. But it really compute loss...And it also can train the net... $\endgroup$ – partida May 22 '17 at 3:16
  • 2
    $\begingroup$ I think the total loss is the sum of all losses in default. You can find an example of the multiple-loss training in the last example of NetTrain->Loss Specifications. You can also specify the total loss as a weighted sum of the individual loss using the third parameter to NetTrain. You can find an example of that in the style transfer example in NetTrain. $\endgroup$ – xslittlegrass May 22 '17 at 4:10
8
$\begingroup$

The default behavior for multiple loss is that they are summed together to give a single total loss as the loss of the network. We can verify that by comparing networks:

  • A network with two losses
  • A network with a single loss that is the sum of the two losses

First, prepare some data

input1 = RandomReal[1, {10, 2}];
input2 = RandomReal[1, {10, 2}];
output1 = RandomReal[1, {10, 10}];
output2 = RandomReal[1, {10, 10}];

This is the network with two losses

SeedRandom[1234];

net1 = NetInitialize[
  NetGraph[{10, Ramp, 10, Ramp, CatenateLayer[], 
    10}, {NetPort["Input1"] -> 1 -> 2 -> 5, 
    NetPort["Input2"] -> 3 -> 4 -> 5 -> 6 -> NetPort["Output2"], 
    2 -> NetPort["Output1"]}, "Input2" -> 2, "Input1" -> 2]]

enter image description here

When training, two MeanSquaredLossLayer's are attached automatically. We can print the loss in training

data1 = <|"Input1" -> input1, "Input2" -> input2, "Output1" -> output1,
    "Output2" -> output2|>;
NetTrain[net1, data1, MaxTrainingRounds -> 4, 
  TrainingProgressFunction -> (Print[#RoundLoss] &)];

(*
0.756067
0.752072
0.748098
0.74413
*)

Now let's compare a network with a single loss that is the sum of the two losses in the first network

SeedRandom[1234];

net2 = NetInitialize[
  NetGraph[{10, Ramp, 10, Ramp, CatenateLayer[], 10, 
    MeanSquaredLossLayer[], MeanSquaredLossLayer[], ReshapeLayer[{1}],
     ReshapeLayer[{1}], 
    TotalLayer[]}, {NetPort["Input1"] -> 1 -> 2 -> 5, 
    NetPort["Input2"] -> 3 -> 4 -> 5 -> 6 -> 8, 2 -> 7, 
    NetPort[7, "Loss"] -> 9 -> 11, 
    NetPort[8, "Loss"] -> 10 -> 11 -> NetPort["Loss"], 
    NetPort["Target1"] -> NetPort[7, "Target"], 
    NetPort["Target2"] -> NetPort[8, "Target"]}, "Input2" -> 2, 
   "Input1" -> 2]]

And we see that the loss are identical to the first network

data2 = <|"Input1" -> input1, "Input2" -> input2, 
   "Target1" -> output1, "Target2" -> output2|>;

NetTrain[net2, data2, MaxTrainingRounds -> 4, 
  TrainingProgressFunction -> (Print[#RoundLoss] &)];

(*
0.756067
0.752072
0.748098
0.74413
*)

The default behavior can be overwritten by setting the third argument of NetTrain.

$\endgroup$
  • $\begingroup$ why use CatenateLayer[] other than TotalLayer[] in the last layer of net2? $\endgroup$ – partida May 24 '17 at 2:13
  • $\begingroup$ @partida You are right, using TotalLayer layer is more reasonable. $\endgroup$ – xslittlegrass May 24 '17 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.