Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.
22
votes
4answers
737 views
Aborting evaluation when the memory exceeds a certain limit
Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent ...
18
votes
2answers
752 views
Debugging memory leaks
I've written my incredibly complex, incredibly elegant analysis function, that works great on small test data. But when I run it on my real (bigger) data set it keeps running out of memory. It turns ...
23
votes
4answers
592 views
How to find out the (current) free memory on the system?
Is there any way in Mathematica to find out the currently free memory on the system Mathematica runs on (like the utility free shows on the command line)? I've ...
21
votes
3answers
613 views
How to place an image inside a notebook, with the minimum memory footprint?
When creating a notebook with images, like photos, the file tends to quickly get big, which makes it difficult to use the notebook format for training material containing a lot of images.
Are there ...
8
votes
2answers
474 views
How do you deal with very large datasets in Mathematica?
This question is in two related parts. The first is about dataset size, the second data wrangling.
Part 1:
I've built a naive Bayesian classifier on Mathematica. It uses ...
14
votes
3answers
357 views
Memory Leak in Frontend - anyone know a workaround?
When using Rasterize on a Graphics expression composed of many differently coloured Graphics objects memory is allocated in the frontend which is then not released or reused when a similar graphic ...
15
votes
4answers
776 views
Plotting large datasets
I have a lot of data (e.g., 64x8192), to plot and it consumes too much memory (more than 2 GB) causing Mathematica to return "no memory available".
For example,
...
7
votes
2answers
164 views
Clean up list of partial outputs for memory
Context
I run long Mathematica sessions; sometimes I have Output up to Out[3000+]. Hence
I would like to be able to clear out some previous outputs which do use lots of memory.
Attempt
I am aware ...
7
votes
1answer
319 views
Memory not freed after running Clear when using a Table
When I'm creating large lists with Table I see the memory footprint increasing.
...
12
votes
3answers
297 views
How to clear parts of a memoized function?
I have a function of two variables, e.g.:
f[a_, b_] := f[a, b] = something f[a - 1, b - 1] etc
With the above code I used the concept of memoization to speed up ...
11
votes
1answer
263 views
Efficient code for tallying entries in very large lists
I am dealing with data sets containing tens of millions of (hashable) entries and simply using the Tally function to count the frequency of each unique list element ...
21
votes
1answer
2k views
Finding long strings of identical digits in transcendental numbers
Introduction
Describing the three main streams of present-day mathematical philosophy (formalism, Platonism and intuitionism) in a well-known book, The Emperor's New Mind, R. Penrose says:
...it ...
11
votes
1answer
313 views
Space-efficient null space of sparse array
I have a roughly 100,000 × 3,000 matrix (as a SparseArray) that I'd like to find the kernel (null space) of.
It has about 500,000 nonzero entries, all -1 or 1.
...
7
votes
1answer
130 views
Mathematica memory management for large arrays
I have come across a weird phenomenon in Mathematica when dealing with large arrays. When generating a list with all the possible subsets of three elements of another list (thus having elements which ...
7
votes
2answers
392 views
Working with large data, pass-by-reference, compiled functions
I think I know the answer to this, but after two days of Googling, this is my last effort before going down an unwanted path...
I have a function that takes lots of large data as arguments as well as ...
5
votes
1answer
287 views
Why does this simple program leak memory?
I have a simple Mathematica program which writes some plots to image files for later conversion into a movie. Unfortunately, the program leaks so much memory that it quickly exhausts all 12G of RAM ...
2
votes
2answers
86 views
Storing result from nested Table in a File for further computations
Consider the following:
Table[Table[x = 20; y = 30; 2 i x + 3 j y , {i, 10}], {j, 3}]
Suppose I want the result of this nested Table to be stored to an ...
1
vote
1answer
284 views
NDSolve runs out of memory
I need to solve a second order ODE numerically. The ODE depends on two parameters (a,b). Things work fine when 'a' is small, but for large 'a' the solutions are oscillating rapidly and Mathematica ...
