3
$\begingroup$

I would like to test optimization for neural networks for that I need to access the weights/bias of the different layers and the derivatives of the output or a loss function with respect to these coefficients? Is there obvious way to do this using built-in functions? At some point

NetTrain

must access the coefficients, compute gradients and re-update them. I am aware of the function

NetInformation[network, "Arrays"]

which returns an association that gives the different coefficients. With some manipulation, eg:

Values@NetInformation[trained, "Arrays"]

it is possible to transform this output in an array that then can go inside of an optimizer code but this may not be the optimal way to do it. Moreover, I don't see in the documentation any way to compute gradients. Also, what would be a good way to replace coefficients in the network?

$\endgroup$
2
  • 1
    $\begingroup$ Look up the documentation for NetPortGradient - that is what you want $\endgroup$
    – M.R.
    Commented Sep 23, 2018 at 19:27
  • $\begingroup$ @M.R. You should add that as an answer, it's the correct answer and limited extra information is needed $\endgroup$
    – Carl Lange
    Commented Oct 10, 2018 at 9:25

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.