I would like to test optimization for neural networks for that I need to access the weights/bias of the different layers and the derivatives of the output or a loss function with respect to these coefficients? Is there obvious way to do this using built-in functions? At some point
NetTrain
must access the coefficients, compute gradients and re-update them. I am aware of the function
NetInformation[network, "Arrays"]
which returns an association that gives the different coefficients. With some manipulation, eg:
Values@NetInformation[trained, "Arrays"]
it is possible to transform this output in an array that then can go inside of an optimizer code but this may not be the optimal way to do it. Moreover, I don't see in the documentation any way to compute gradients. Also, what would be a good way to replace coefficients in the network?
