4
$\begingroup$

I have to compute the covariance of 50 very large integer matrices (2500x2000 elements). However, according to my estimation this will take around 10 days. Do you have any ideas how to speed things up?

Improve this question
$\endgroup$
7
  • $\begingroup$ A 5000x5000 matrix calculates under 10s for me Covariance[RandomReal[1, {5000, 5000}]]; // AbsoluteTiming What type of elements do you have? $\endgroup$
    – ssch
    Commented Nov 14, 2013 at 20:58
  • $\begingroup$ How did you estimate it will takes 10 days per matrix? (btw, 10 days using which type of computer? since the CPU type and amount of RAM should make some difference) $\endgroup$
    – Nasser
    Commented Nov 14, 2013 at 21:01
  • $\begingroup$ @Nasser Sorry, I made a mistake; it's 10 days altogether. I compute the covariance of 100 smaller, and scaled everything assuming linearity. $\endgroup$
    – user9234701
    Commented Nov 14, 2013 at 21:10
  • 1
    $\begingroup$ @phidelio Make sure your matrix contains only machine precision numbers, i.e. use N[matrix] instead of matrix. $\endgroup$
    – Szabolcs
    Commented Nov 14, 2013 at 21:26
  • 1
    $\begingroup$ @phidelio Then you have your solution---use N on them. Otherwise Mathematica will attempt to calculate an exact solution, which is very slow and likely not what you need. $\endgroup$
    – Szabolcs
    Commented Nov 14, 2013 at 21:36

1 Answer 1

Reset to default
10
$\begingroup$

If the matrix contains exact integers, Mathematica will compute an exact result (in terms of exact rational numbers). This is very slow.

If you convert your matrix to (inexact) machine precision numbers, the calculation will be much much faster.

Use Covariance@N[matrix] instead of Covariance[matrix].

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, I will try it tomorrow and let you know (I don't have Mathematica on this computer). $\endgroup$
    – user9234701
    Commented Nov 14, 2013 at 22:14
  • $\begingroup$ This definitely works. Thanks a lot. $\endgroup$
    – user9234701
    Commented Nov 15, 2013 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.