I am following the artificial neural networks literature and apparently the latest trend is to use the rectified linear units (ReLU) as the activation functions for each neuron. I tried to take the derivative of this function in Mathematica but it gives indeterminate at x=0:
f[x_]:= Max[0,x] D[f[x],x]
In the neural network computation, one explicitly defines the derivative of the ReLU at x=0 as 0. Can we also instruct Mathematica to do the same? Do we have to define the derivative at x=0 explicitly somehow? Or there is another trick here? Since Mathematica 11 now has the deep learning tools, I am assuming that this problem must have been addressed there?