I have a loop which computes a table, tab
, of results at each iteration. This table is usually very large and I would like to use PutAppend
or anything of the sort to save the data in a file and therefore saving memory. An example would look like the following:
Put["test.csv"];
Do[tab=RandomReal[{0,1},10];PutAppend[tab, "test.csv"],{5}]
When I import the data in test.csv
for later use I do this via data=Flatten[Import[".../test.csv","Data"],1]
. However, the list data
is not ready to use because typical elements look e.g. like
In[45]:= data[[1]]
Out[45]= "{0.3105121460958056"
In[43]:= data[[2]]
Out[43]= 0.0657094
In[44]:= data[[4]]
Out[44]= " "
Which is a mixture of strings and reals.
So my question is: how can I append the table tab
at each iteration of the loop to a file which later is ready to use as a list of rank two of reals?
Thanks for your help.