How to train each layer in a Neural Network so they optimize different loss functions in an adversarial network?

Question

Example NetGraph to illustrate the idea (Input is an online signal with a value between -1 and 1, Noise is Gaussian Centered at 0 and standard deviation 0.1, EvilNet is constrained to output a value between Input-0.2 and Input+0.2):

I am trying to train "GoodNet" in the network in a way that minimizes "GoodLoss", while training "EvilNet" in a way that minimizes "EvilLoss" (maximizing "GoodLoss").

I have tried using TrainingUpdateSchedule, but I don't find a way to indicate which Loss Function should be minimized at each step.

I have also tried alternating calls to NetTrain with different parameters (freezing GoodNet and minimizing EvilLoss, and then freezing EvilNet and minimizing GoodLoss), but the process is very inefficient, since the momenta for the ADAM method are reset each time NetTrain is called, and there is also significant overhead.

Here is some code about the Net's specification (note that this is not the code I am using in my application, but rather a trimmed down version to deal with the essentials of the question).

trygoodnet = NetChain[{
   BatchNormalizationLayer[],
   NetModel[{"Self-Normalizing Net for Numeric Data", 
     "TaskType" -> "Regression", "Depth" -> 6, "Width" -> 8, 
     "DropoutProbability" -> 0}]
   }]
trybadnet = NetGraph[<|
   "MainChain" -> NetChain[{
      BatchNormalizationLayer[],
      NetReplacePart[
       NetModel[{"Self-Normalizing Net for Numeric Data", 
         "TaskType" -> "Regression", "Depth" -> 6, "Width" -> 8, 
         "DropoutProbability" -> 0}], {"Input" -> 1}]
      }],
   "Limiter" -> ElementwiseLayer[(LogisticSigmoid[#] - 1/2)*0.4 &],
   "Total" -> TotalLayer[]
   |>,
  {NetPort["Input"] -> "MainChain" -> "Limiter" -> "Total", 
   NetPort["Input"] -> "Total"}]

TryCleanNet = 
 NetGraph[<|"EvilNet" -> trybadnet, "GoodNet" -> trygoodnet, 
   "NoisyInput" -> TotalLayer[], "RecInput" -> TotalLayer[], 
   "MLSL" -> MeanSquaredLossLayer[], 
   "-1" -> ElementwiseLayer[-# &]|>, {NetPort["Input"] -> 
    "EvilNet" -> 
     "RecInput" -> "GoodNet" -> "MLSL" -> NetPort["GoodLoss"], 
   NetPort["Input"] -> "NoisyInput" -> "RecInput", 
   NetPort["Noise"] -> "NoisyInput", NetPort["Input"] -> "MLSL", 
   "MLSL" -> "-1" -> NetPort["EvilLoss"]}]

Here is the best attempt I have found so far (minimizing GoodLoss, but giving a negative LearningRateMultiplier to the EvilNet)

TrainedCleanNet = 
 NetTrain[TryCleanNet, 
  Function[Block[{pick = ({#} & /@ 
        RandomVariate[
         TruncatedDistribution[{0, 1}, 
          MaxwellDistribution[0.4]], #BatchSize]), 
     noise = ({#} & /@ 
        RandomVariate[NormalDistribution[0, 0.1], #BatchSize])}, <|
     "Input" -> pick, "Noise" -> noise|>]], 
  TimeGoal -> Quantity[40, "Hours"], LossFunction -> {"GoodLoss"}, 
  TrainingUpdateSchedule -> {"GoodNet", "EvilNet"}, 
  LearningRate -> 10^-4, 
  LearningRateMultipliers -> {"GoodNet" -> 1, "EvilNet" -> -5}, 
  BatchSize -> 200000]

That solution works well for the specific case presented here, but it doesn't generalize as well as being able to assign each network for optimizing different losses.