Fast computation of eigenvalues of large Hermitian matrices

I need to compute numerically all the eigenvalues of roughly $500$ large Hermitian matrices, $(5 \times 10^5) \times (5 \times 10^5)$).

At best, I can speed up my computation such that it takes almost twice the amount of time that I can reach on python using the numpy package.

I would like if such a difference can be in general further reduced, and the two performance can be similar, or there is a gap between mathematica an python that I cannot fill.

• That's weird. I thought that both numpy and Mathematica link to the same library (ARPACK). Be sure to use Method->"Arnoldi"; see also documentation of Eigenvalues. – Henrik Schumacher Mar 2 '18 at 8:17
• Oh thank you. I will try to do some tests on that. But so, on general ground, should I expect to reach the same performance with mathematica and python? thatnks! – Dario Rosa Mar 2 '18 at 8:19
• Mathematica should perfom some additional copy operations for the transfer to and from the library in order to preserve immutability. But that should be negligible compared to the effort of computing the eigenvalues. Once the matrix is in the library (and the same options are uses), I see now reason why there should be a performance difference. Unfortunately, Mathematica provides only few suboptions for "Arnoldi" to tune it further. Moreover, I don't know to which version of ARPACK they link. – Henrik Schumacher Mar 2 '18 at 8:26
• I see. Actually from the documentation it seems that Arnoldi method can be used only if we need few eigenvalues but I need all of them. Do you know whether I can overcome this problem? – Dario Rosa Mar 2 '18 at 8:30
• Doh. Yeah, that is an issue. You can try to compute first and secand half of spectrum separately. Moreover, it just came to my mind that Mathematica might not link to the parallel version of ARPACK =/ – Henrik Schumacher Mar 2 '18 at 8:35