# Is the stochastic gradient descent minimization implemented in Mathematica?

I want to use Stochastic Gradient Descent method on its own (not for neural networks) to minimize an objective function in Mathematica. I couldn't find any SGD routine implemented in Mathematica (12.0) so far. Is it an option in one of the routines? Thanks.

• FindMinimum uses a gradient for its various methods, but I haven't seen stochastic gradient descent there. Probably when a full gradient is available it's not that effective compared to the others. You'd normally use SGD for parameter estimation / regression, when the cost surface is unavailable but you have an approx gradient at random sample points. In any case, here's a Wolfram Demonstrations Project notebook from 2011 demonstrations.wolfram.com/StochasticGradientDescent . There's also BayesianMinimization which can minimize non-deterministic functions if that interests you. – flinty Jul 2 at 21:53
• @flinty, in my case the cost function is exactly in the form of first equation in en.wikipedia.org/wiki/Stochastic_gradient_descent and SGD seems a very suitable method to minimize this function. My problem has not come from neural network. So even though the neural network related commands in Mathematica uses SGD, I don't know how to use this method for my own function. – dbm Jul 2 at 21:59
• @dbm It could be better to show your function and your attempt to minimize it. – Alex Trounev Jul 11 at 12:45