4
$\begingroup$

I am trying to initialize a 48600 x 48600 packed array from a sparse matrix and a diagonal matrix in Mathematica 8. If S1 is a sparsearray of that size then and S2 is list of 48600 length, then I am using the command

H=Developer`ToPackedArray[S1+DiagonalMatrix[S2]];

This returns an error.

General::nomem:
   The current computation was aborted because there was insufficient memory
    available to complete the computation.

This is impossible because the RAM I have is 512GB, plus when I do the same thing in a 256GB Mathematica 11.0.2 machine, it goes through, but then I need to perform other operations(diagonalization, multiplication) on this so I actually need the 512 GB RAM.

Additionally the same machine of 512GB RAM returns an error when I try to make the sparsearry go to normalform and then pack it, as,

In:-S1packed=Developer`ToPackedArray[Normal[S1]];

SparseArray::ntb:


 Cannot convert the sparse array SparseArray[<463320>, {48620, 48620}]
     to an ordinary array because the 2363904400
     elements required exceeds the current size limit.

Out[1]= SystemException[SparseArrayNormalLimit,

>    Normal[SparseArray[<463320>, {48620, 48620}]]]

Even this operation goes through in Mathematica11 256 GB RAM, but not here. Apart from the obvious answer install Mathematica 11 , which I cant since its an institute machine, anybosy know anything about increasing max memory usage/size limit of normalform from SparseArray in Mathematica?

Update:- I was testing both the systems with

T1=RandomReal[1,{50000,50000}];

The 512 GB Mathematica 8 machine gives,

RandomReal::arrsize: 
   In array dimensions {50000, 50000} at position 2 of 
    RandomReal[1, {50000, 50000}], the product of the first 2 dimensions is 
    2500000000, which is not a machine integer.

It's a 64 bit machine so it's impossible that the largest machine integer is actually less than this.

While for the 256GB Mathematica 11.0.2 machine it goes through(takes a lot of RAM though). What's going on?

$\endgroup$
8
  • 5
    $\begingroup$ Hm. That's odd. In Mathematica 12, I am able to execute A = RandomReal[{-1, 1}, {48600, 48600}]; without any problems even with only 16 GB of RAM installed. The matrix requires about 18 GB to be stored, so of course quite a lot of swapping is going on. But it works. $\endgroup$ Oct 25, 2020 at 7:39
  • $\begingroup$ So I guess there was a problem in the older version of Mathematica, it forces itself to assume integers are 32 bit even on a 64 bit machine or something? I guess only way out is then installing a newer version? $\endgroup$ Oct 25, 2020 at 7:47
  • $\begingroup$ Well, Mathematica 8 was released in 2010. The kernel was supposed to support 64bit, but probably some library was not up to date. Or just this saveguard for the 32bit check had not been removed although it did not matter anymore at that time. Indeed, the best method would be to have a more recent version installed on the machine. $\endgroup$ Oct 25, 2020 at 7:50
  • $\begingroup$ Would it help to encapsulate everything in a Block construct with infinite extra precision? Try: Block[{$MaxExtraPrecision=Infinity}, . . .] $\endgroup$
    – Dominic
    Oct 25, 2020 at 11:32
  • $\begingroup$ You don’t say enough important detail: Operating system under which runs Mathematica. Do you have access to the administrator resource. If I were in your place, and I have the option I boot the computer "512 go" on a bootable USB key win 10. I download Mathematica 12 trial versions valable 1 month. I would come out of the environment I’m trapped in. $\endgroup$ Oct 25, 2020 at 13:12

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.