I want to use Stochastic Gradient Descent method on its own (not for neural networks) to minimize an objective function in Mathematica. I couldn't find any SGD routine implemented in Mathematica (12.0) so far. Is it an option in one of the routines? Thanks.

  • 2
    $\begingroup$ FindMinimum uses a gradient for its various methods, but I haven't seen stochastic gradient descent there. Probably when a full gradient is available it's not that effective compared to the others. You'd normally use SGD for parameter estimation / regression, when the cost surface is unavailable but you have an approx gradient at random sample points. In any case, here's a Wolfram Demonstrations Project notebook from 2011 demonstrations.wolfram.com/StochasticGradientDescent . There's also BayesianMinimization which can minimize non-deterministic functions if that interests you. $\endgroup$ – flinty Jul 2 at 21:53
  • 2
    $\begingroup$ @flinty, in my case the cost function is exactly in the form of first equation in en.wikipedia.org/wiki/Stochastic_gradient_descent and SGD seems a very suitable method to minimize this function. My problem has not come from neural network. So even though the neural network related commands in Mathematica uses SGD, I don't know how to use this method for my own function. $\endgroup$ – dbm Jul 2 at 21:59
  • $\begingroup$ @dbm It could be better to show your function and your attempt to minimize it. $\endgroup$ – Alex Trounev Jul 11 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.