Err... When I try to Import a .dat file by
dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat",
"Table"];
or
dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat"];
mathematica run out of memory and crashed the kernel,I admitted that the dat file is a bit large ~38Mb, however, it shouldn't have been a problem considering my 2G RAM. And when I checked the progresses by Process Explorer, I found that the Private Bytes of one MathKernel.exe grows rapidly from 63Mb all the way to more than 700Mb when I evaluate the code dat = Import[...] listed above. How could this happen?Help!!!!!
the omni_m_all_years.dat file can be download under ftp://cdaweb.gsfc.nasa.gov/pub/data/omni/low_res_omni/
(in fact you can Import
the dat file directly from website if your Mma has good connection with internet)
and I use Mathematica9.0 under Windows7
ReadList
, instead. $\endgroup$Import
functionality is too general and tries to be too smart (that is auto-detect almost everything) and never has been decently optimized. The usual workaround, especially for text formats, is to use one of the lower level functions to write something specialiced as others have suggested and shown... $\endgroup$