1
$\begingroup$

I have 16 NxN matrices, where N can be anywhere between 1000-64000.. Some of them are real and some of them are complex. I want to Reshape then to form a 4Nx4N matrix. When I try to accomplish this using ArrayFlatten or Flatten[,{{1,3},{2,4}}], this results in a huge spike in memory utilization. For N=2000, that is around 6GB RAM. So for larger N it is going to be astronomically large. However, when I checked the space required to store a RandomComplex[] matrix of the same size was around 2GB, which is still a bit high but understandable with 128bit complex numbers.

I understand that this is probably due to Mathematica unpacking the array while flattening, which is what I get when I check with PackedArrayQ[] which returns false for the final array.

My question is is there any way to keep the array packed during flattening? My idea was the unpacking occurs probably because of different data types, is that the case? If so, then how do I ensure all of them are complex and the resulting matrix remains packed.

I need the matrix in that specific format because I need to calculate inverse and eigenvalues later. Any suggestions which involve not using Flatten but get the inverse of the matrix and eigenvalues are also appreciated.

$\endgroup$
1
$\begingroup$

I guess some of the matrices are SparseArrays? Then in order to prevent the SparseArrays from being converted to dense matrices, make sure that all matrices you want ArrayFlatten are SparseArrays with the same background element. 0, 0., and 0. + 0. I are not the same in this respect. You might have to convert also dense matrics to SparseArray.

If you flatten only dense arrays, you also have to make sure first that all matrices are of the same machine number type: Either all Integer, double recision reals, or double precision complex numbers.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Unfortunately,none of the matrices are Sparse so yes I am flattening only dense arrays. How do I ensure all of them are of machine number first? I add +0 I by hand to all the real matrices? $\endgroup$ – Roopayan Ghosh May 7 at 13:12
  • $\begingroup$ Thanks, That worked, But what basically goes on? What extra information does Mathematica store that this becomes so large? $\endgroup$ – Roopayan Ghosh May 7 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.