There comes a new paper on arXiv (arXiv 1912.01412, Deep Learning for Symbolic Mathematics by Guillaume Lample, François Charton) claiming that a deep learning model outperforms Mathematica a lot in symbolic integration and ODE solving. The comparison in success rate is the following (30s timeout for MMA evaluation while the new method takes typically 1s). Beam size roughly means the allowed maximal number of solutions found (not necessarily all correct). If one of them is correct, it's enough since verifying a solution is way simpler. This is what it means by a successful case in this new method.
The method is not complex.
Then, they treat such a sequence of an integral or ODE as a sentence to be translated to another sequence (the solution), in much the same way as in machine translation.
They propose three ways to generate the training sets,
- Forward (use MMA to integrate randomly generated stuff)
- Backward (use MMA to differentiate randomly generated stuff)
- Integration By Parts (use data accumulated via the above two to generate more data with integration by parts)
I was wondering if this really surpasses the state-of-art MMA in symbolic integration and ODE solving as they have claimed. Compared to the generality of MMA, is their problem-solution space biased in any way and why? If somewhat biased, why MMA fails to manage some of their problem-solution? These concerns are algorithmically well-defined and thus not opinion-based in any sense, I feel.
If not very biased, this seems to be at least partially what one would expect in a next-generation MMA. Or is it possible to realize in the current version of MMA?