Does Mathematica have any AD functionality or does it only support symbolic differentiation? If not, are there any packages or other third party implementations available?
Edit (J. M.)
Here is an example that might clarify how AD works. Consider the Horner method for evaluating a polynomial from its coefficients:
h[coeffs_?VectorQ, x_] := Module[{np1 = Length[coeffs], p, k},
p = 0;
Do[p = x p + coeffs[[k]], {k, np1, 1, -1}];
p
]
An AD method should be able to "differentiate" this procedure and automatically produce this:
hp[coeffs_?VectorQ, x_] := Module[{np1 = Length[coeffs], p, p$, k},
p$ = 0; p = 0;
Do[p$ = x p$ + p;
p = x p + coeffs[[k]], {k, np1, 1, -1}];
p$
]
Note the use of the product rule to differentiate the line p = x p + coeffs[[k]]
, as well as the ordering of the variable assignments in the "differentiated procedure".
There are more complicated examples, but traditionally, AD methods have dealt with routines that have been written procedurally. A challenge, then, is if AD methods can cope with more "idiomatic" Mathematica programs; that is, functional/rule-based algorithms. I believe that will be more difficult.