4
$\begingroup$

Jointly training a neural network to solve multiple tasks simultaneously sometimes results in elevated accuracy, and I'm looking for any known examples of this in Mathematica.

Here's the specific problem I'm trying to formulate into the NetGraph/NetTrain architecture: I have a set of facial images and three labels for each image: age (integer) and gender (binary) and race (7 classes).

Using Keras, training jointly on data with multiple labels per sample is straightforward using a base network with multiple final outputs and losses, e.g. cross entropy loss for race and mean absolute loss for age.

I'm not sure how to approach this in Mathematica, and any high level guidance would be appreciated, but specifically, how can one tell NetTrain to use different losses for different sublabel data?

Improve this question
$\endgroup$
3
$\begingroup$

Step 1 is to ensure that your network maps the output of each of your desired regression objectives to a named NetPort (e.g. NetPort["AgeOut"]). Your classification objective should pass through a soft max layer prior to going to the output, and the binary classification objective should pass through a sigmoid elementwise layer. The integer objective can remain a scalar. If you follow these rules, then NetTrain will automatically attach the appropriate loss layers to each objective.

Step 2 is to use an association in your NetTrain statement. For example, <|"Input"->{face1,face2,...},"AgeOut"->{25,31,...},"GenderOut"->{0,1,...}|>.

If you do this, the network should attempt to simultaneously minimize each loss.

Improve this answer
$\endgroup$
  • $\begingroup$ So the network attaches appropriate layers, but how would I override that or manually configure that? Also are custom losses allowed? $\endgroup$ – M.R. Aug 9 '17 at 1:05
  • $\begingroup$ Yes, you can define a custom loss layer as a NetGraph with inputs "Input" and "Target" and outputs "Loss", and then use the loss syntax as defined in the NetTrain docs to attach it to your output. The custom loss layers should have a scalar output that is a function to be minimized. $\endgroup$ – Yss Aug 9 '17 at 1:42
  • 2
    $\begingroup$ @M.R. you can use the third argument of NetTrain to attach specific weights and loss layers to specific output ports. See the "Details and Options" section of NetTrain for more info on that, but as an example, you could write e.g. {"AgeOut" -> MeanAbsoluteLossLayer[], "SexOut" -> CrossEntropyLossLayer["Indexed"] -> Scaled[5]}. $\endgroup$ – Taliesin Beynon Aug 16 '17 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.