4
$\begingroup$

Possible Duplicate:
Save interpolated function

I am importing tables with sizes of 100k - 500k rows and 4 columns. The larger the table, the slower my Mathematica runs for all computations after the import. I import the table and then apply Interpolation to them. After that the tables are no longer required and I use just the Interpolation functions.

So my question is, can I somehow restart the kernel to get rid of the large data tables but keep the interpolation functions calculated from those tables? Or is some other solution possible? THanks!

Improve this question
$\endgroup$
4
  • $\begingroup$ You don't need to restart. You can just throw away the table using Clear, ClearAll or Unset. $\endgroup$
    – Sjoerd C. de Vries
    Commented Oct 25, 2012 at 23:19
  • $\begingroup$ @SjoerdC.deVries, in my experience the Clear, ClearAll, and Unset functions do nothing to stop Mathematica from hogging all of your memory. $\endgroup$
    – user4368
    Commented Oct 25, 2012 at 23:23
  • $\begingroup$ @Jason, did you try to set $HistoryLength = 0? $\endgroup$
    – user21
    Commented Oct 25, 2012 at 23:25
  • $\begingroup$ (am I hijacking the thread here?) Yeah, I tried all sorts of memory-reducing tricks. Basically I would generate tables of data in mathematica and save them to .m files which would be between 200 and 600 megabytes. Then I would quit, and the act of importing the data file would nearly take all the RAM I had, and would take a long time. Then I switched to using the netcdf data format, the files are a fourth the size and importing is nearly instantaneous. $\endgroup$
    – user4368
    Commented Oct 25, 2012 at 23:28

1 Answer 1

Reset to default
3
$\begingroup$

You can definitely save an interpolating function. A simple example is 

temp = Table[40 Cos[x/100] + RandomReal[], {x, 1000}];
tempfunc = Interpolation[temp];
tempfunc >> "tst.m";
Quit

Then when you input

tempfunc=<<"tst.m";

you get back your interpolation function. But I have doubts as to whether this will speed up any computations. The file "tst.m" has, in ascii format, all the information that was in the original array.

What format are the data files in that you import? I had a problem recently where Mathematica would nearly take up all the memory in my system importing a couple of data files. Then I started saving them using the netcdf format and it became a lot less memory intensive.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I've done function calls on interpolation functions that have used small and large data arrays and have seen no difference in their time. How can I convert to netcdf? I am importing 30 text files of about 16 MB each. $\endgroup$
    – BeauGeste
    Commented Oct 25, 2012 at 23:39
  • $\begingroup$ @Jason whats the difference between this and Export? I have always used Export to save my interpolating function as a .mat file $\endgroup$
    – dearN
    Commented Oct 26, 2012 at 0:08
  • $\begingroup$ @drN The >> above is shorthand for the Put function, which writes expressions to a plain text file. If you want to write to a binary file format, like .mat or .nc, you have to use Export. $\endgroup$
    – user4368
    Commented Oct 26, 2012 at 20:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.