11
$\begingroup$

There comes a new paper on arXiv (arXiv 1912.01412, Deep Learning for Symbolic Mathematics by Guillaume Lample, François Charton) claiming that a deep learning model outperforms Mathematica a lot in symbolic integration and ODE solving. The comparison in success rate is the following (30s timeout for MMA evaluation while the new method takes typically 1s). enter image description here Beam size roughly means the allowed maximal number of solutions found (not necessarily all correct). If one of them is correct, it's enough since verifying a solution is way simpler. This is what it means by a successful case in this new method.

The method is not complex.

  1. It starts with making mathematical expressions into the data structure of binary tree, which naturally gives a prefix notation sequence.

  2. Then, they treat such a sequence of an integral or ODE as a sentence to be translated to another sequence (the solution), in much the same way as in machine translation.

They propose three ways to generate the training sets,

  • Forward (use MMA to integrate randomly generated stuff)
  • Backward (use MMA to differentiate randomly generated stuff)
  • Integration By Parts (use data accumulated via the above two to generate more data with integration by parts)

I was wondering if this really surpasses the state-of-art MMA in symbolic integration and ODE solving as they have claimed. Compared to the generality of MMA, is their problem-solution space biased in any way and why? If somewhat biased, why MMA fails to manage some of their problem-solution? These concerns are algorithmically well-defined and thus not opinion-based in any sense, I feel.

If not very biased, this seems to be at least partially what one would expect in a next-generation MMA. Or is it possible to realize in the current version of MMA?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You might be able to realize this in the current version of MMA in a way similar to how Rubi rulebasedintegration.org is realized with the current version. A much smaller task would be to compare Rubi and this new method and see if you could discover within the calculations of the new method that you can construct rules or results which are better than Rubi and contribute those to the Rubi project. I had considered this idea when I read the paper. As for Wolfram replacing their decades of work on integration, replacing very large very old very complicated code is often very very bad $\endgroup$
    – Bill
    Dec 18, 2019 at 23:44
  • 1
    $\begingroup$ Read some of it and noted that it needs an external framework to verify which of the many solutions it generates is correct. It finds potential solutions quickly but I did not see mention on how long it took to verify which of the generated solutions solved the equation. Mma was only given 30 seconds to solve. Was a time restriction imposed on the total solve time of the proposed solver (generating the solution set and verifying which were correct)? In any case it is very interesting as there may be some tipping point on which to branch into ML paths over algorithmic solvers. $\endgroup$
    – Edmund
    Dec 18, 2019 at 23:52
  • $\begingroup$ @Bill I don't know, sometimes there are advantages of purging old code, and starting again from scratch. $\endgroup$
    – QuantumDot
    Dec 19, 2019 at 19:46

1 Answer 1

Reset to default
16
$\begingroup$

Not everything is as peachy as claimed in that paper due to limitations of the problems deep learning can approach in their framework. Wolfram Language (WL) addresses much wider range of problems. Also, WL has an excellent machine learning framework and probably if deep learning methods in symbolic math domain will grow up into something solid and general they might get naturally absorbed into internal built-in tools in the future. You can always check what things are trending in deep learning domain by scanning through models at Wolfram Neural Net repository, which follows closely modern research:

https://resources.wolframcloud.com/NeuralNetRepository

Please read a followup review of Lample and Charton paper:

The Use of Deep Learning for Symbolic Integration: A Review of (Lample and Charton, 2019)

https://arxiv.org/abs/1912.05752

Lample and Charton (2019) describe a system that uses deep learning technology to compute symbolic, indefinite integrals, and to find symbolic solutions to first- and second-order ordinary differential equations, when the solutions are elementary functions. They found that, over a particular test set, the system could find solutions more successfully than sophisticated packages for symbolic mathematics such as Mathematica run with a long time-out. This is an impressive accomplishment, as far as it goes. However, the system can handle only a quite limited subset of the problems that Mathematica deals with, and the test set has significant built-in biases. Therefore the claim that this outperforms Mathematica on symbolic integration needs to be very much qualified.

Improve this answer
$\endgroup$
2
  • 9
    $\begingroup$ In particular, from that paper's summary: "The correct statement, as regards integration, is as follows: The transformer model outperforms Mathematica and Matlab in computing symbolic indefinite integrals of enormously complex functions of a single variable ‘x’ whose integral is a much smaller elementary function containing no constant symbols other than the integers −5 to 5." $\endgroup$
    – Ben Kalziqi
    Dec 19, 2019 at 3:15
  • 1
    $\begingroup$ Vitaliy, the idea that mathematics can be treated as a natural language seems right up WRi’s alley. While what @BenKalziqi brings attention to is quite a narrow gap, what, if anything, can be gained from this? Are there a certain subset of operations that can be expedited with this method? What could possibly be gained from this expedited process, with possible additions of a hybridized approached wherein some symbolic input is passed between computational algebraic algorithms and these NLP-symbolic-math neural networks? Glad to see you comment on this! $\endgroup$
    – CA Trevillian
    Dec 21, 2019 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.