# Controlling learning rate decay or cyclic learning rate in Mathematica's neural nets

I am using Mathematica's NetTrain[] function for training neural nets. There is a way to set the learning rate (LearningRate and LearningRateMultiplier options) but I want a decreasing learning rate or changes based on development of the loss.

Current call for transfer learning with fixed learning rate:

NetTrain[
preTrainedNet, trainData, All,
ValidationSet -> valData, MaxTrainingRounds -> epochs, TargetDevice -> "GPU",
LearningRateMultipliers -> {
"classifier" -> lr,
{"base", 1, "conv_conv2d"} -> lr,
{"base", 1, "conv_relu"} -> lr,
_ -> 0},
BatchSize -> 8,
TrainingProgressCheckpointing ->
{"Directory", "C:\\DataSets\\RZ-DL-Aug-Pre", "Interval" -> Quantity[10, "Rounds"]}
];


How can I implement what I need?

Maybe the suboption "LearningRateSchedule" of the option Method helps? See the doc page of NetTrain > Options > Method. But I have to admit that the documentation is quite vague about the meaning of the function that can be fed to "LearningRateSchedule".