In Steven Wolfram's blog entries, he discusses using Mathematica to discover algorithms. I have tried several different Google searches for papers on such topics, and have found none. I suspect this is a case where I'm not using the proper search terms. I could imagine some sort of genetic algorithm based approach that takes a genome and turns that into an algorithm, but I would be curious what specifically he is discussing.
The example I'm familiar with is the claim that some of the formulae used internally by Mathematica's functions were derived using Mathematica itself.
For machine precision most special functions use Wolfram Language-derived rational minimax approximations.
Most of the algorithms in the Wolfram Language, including all their special cases, were explicitly constructed by hand. But some algorithms were instead effectively created automatically by computer.
Many of the algorithms used for machine‐precision numerical evaluation of mathematical functions are examples. The main parts of such algorithms are formulas which are as short as possible but which yield the best numerical approximations.
Most such formulas used in the Wolfram Language were actually derived by the Wolfram Language itself. Often many months of computation were required, but the result was a short formula that can be used to evaluate functions in an optimal way.
We can of course presume that the formulae themselves and how exactly they were generated is a trade secret. ;)
An example of mine, from long ago, is at https://space.mit.edu/home/jpd/LinTrigFunc.html
Fast "spherical regression" used in a space mission.