# Fast export of large numerical data

## Edit

It seems that the bottelneck is not due to eigen value problem. I will esit the question soon.

I have a 6000by6000 matrix which I got after some calculations. I want to get the eigenvalues and eigenvectors of this matrix. However, it seems 24GB of RAM is not enough! Just before getting the eigenvalues and vectors MMA uses 12GB of RAM, and after running Eigensystem it crashes due to insufficient amount of RAM. I thought I could save the results just before the eigen value problem, exit MMA and import them again so that I might have those extra 12GB of RAM used just before eigen value problem.

If I try Export["results.wdx",results] or Export["results.csv",results] it takes so long that it makes me bored! I tried Export["results.h5",results]. It exports the data, the 6000by6000 matrix, fast, but when I import the resulting file there is nothing. Here is an example:

SetDirectory[NotebookDirectory[]];
matrix = RandomReal[{0, 1}, {10^4, 10^4}];
Export["matrix.h5", matrix];
revived = Import["matrix.h5"];


the last line returns:

{"/Dataset1"}


In the document it mentions that:

Import["file.h5",elem] imports the specified element from an HDF5 file


which I cannot understand. How can I export my data in h5 format and import it again? Also, it would be good if someone give me a solution for the eigen value problem. How can I get the eigenvalues and eigenvectors of a 6000by6000 matrix without running out of RAM? I use MMA 9 and ByteCount for the 6000by6000 matrix returns 800 mega byte.

• For the first question, you need revived = Import["matrix.h5", "Data"]. – xzczd Dec 15 '14 at 13:26
• Thank you it worked. I will edit the question to reflect the second question. I should have read the document more carefully! – MOON Dec 15 '14 at 13:35
• Related: mathematica.stackexchange.com/questions/23944/…. According to the answer, you should need just 0.26 GB for your matrix. Are you using numerical matrices, or your problem involves symbolic expressions? – Pincopallino Dec 15 '14 at 14:34