Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.
5
votes
1answer
36 views
How to relate memory usage with occupied positions of SparseArrays?
What is the relation of memory usage of a SparseArray
and the number of its occupied positions?
Let's say you build a 100.000.000 by 10 element SparseArray.
And fill the two position 1/1 and ...
1
vote
0answers
55 views
FindRoot runs out of memory in Windows but not on Mac
I am using FindRoot to find solutions to a large, highly oscillatory equation which I generated using other code. When I run ...
1
vote
0answers
82 views
Getting MemoryAllocationFailure from NIntegrate
When numerically calculating a double integral using NIntegrate over an infinite domain (with all options at their default), Mathematica 7 calculates my integral ...
0
votes
0answers
41 views
Is it possible to selectively save the calculated values (array) at memory?
My calculation might reach at the limit of my computer's memory because so many output values like the array of a[i,j,k] were produced in the calculation.
In order to retain memory, can be the ...
2
votes
0answers
156 views
Is mathematica storing information it shouldn't store?
I'm seeking to find solutions to a numerical integration with a large set of parameter combinations (basically, I'm doing a brute parameter sampling). Yet, I believe the memory of the computer is ...
5
votes
1answer
288 views
Why does this simple program leak memory?
I have a simple Mathematica program which writes some plots to image files for later conversion into a movie. Unfortunately, the program leaks so much memory that it quickly exhausts all 12G of RAM ...
2
votes
1answer
50 views
PolynomialRemainder memory
This calculation makes the kernel crash because it needs so much memory. Thoughts on how to get around this?
...
3
votes
0answers
65 views
Is it possible to lower the bitwidth of (floating point) numbers?
The answer to this question will most probably be a definite no. Nevertheless, I would like to know whether it is possible to represent reals in the memory on less than the standard 32 or 64 bits ...
2
votes
0answers
121 views
Project Euler (memory and speed issues)
Let n be a positive integer. An integer triple (a, b, c) is called a factorisation triple of n if:
1 ≤ a ≤ b ≤ c
a·b·c = n.
Define f(n) to be a + b + c for the factorisation triple (a, b, c) of n ...
4
votes
0answers
77 views
Some memory issues with In, Out and HistoryLength
I was trying to say something useful here
Debugging memory leaks
but I got very confused. Can somebody please confirm that the following will cause a big gap between the allocated memory you start ...
7
votes
1answer
132 views
Mathematica memory management for large arrays
I have come across a weird phenomenon in Mathematica when dealing with large arrays. When generating a list with all the possible subsets of three elements of another list (thus having elements which ...
2
votes
2answers
87 views
Storing result from nested Table in a File for further computations
Consider the following:
Table[Table[x = 20; y = 30; 2 i x + 3 j y , {i, 10}], {j, 3}]
Suppose I want the result of this nested Table to be stored to an ...
0
votes
1answer
195 views
Why does memory run out when adding same-sized matrices in a loop?
I have written a module that it generates a 101x101x51 zero matrix at first. At every iteration, it generates a list by funct1 and based on this list, if list is empty it gives the previous result. ...
12
votes
3answers
300 views
How to clear parts of a memoized function?
I have a function of two variables, e.g.:
f[a_, b_] := f[a, b] = something f[a - 1, b - 1] etc
With the above code I used the concept of memoization to speed up ...
11
votes
1answer
270 views
Efficient code for tallying entries in very large lists
I am dealing with data sets containing tens of millions of (hashable) entries and simply using the Tally function to count the frequency of each unique list element ...
5
votes
1answer
144 views
Processing and storing a graph with a large number of vertices s.t. k-hop neighborhoods are efficiently retrievable
I have an unweighted, undirected graph $G$ with $(v_1, ..., v_N) \in V$ vertices, where each vertex in the lattice represents a string of some length ($\leq 200$ characters), and I need to construct ...
0
votes
0answers
68 views
Is there a way to ask Mathematica for the RAM installed? [duplicate]
Possible Duplicate:
How to find out the (current) free memory on the system?
Do we kwow of a function to get the installed
amount of physical system memory RAM of my Windows PC?
0
votes
0answers
70 views
How to set Mathematica for a max system memory usage? [duplicate]
Possible Duplicate:
Aborting evaluation when the memory exceeds a certain limit
Is there a preference setting to restrict the use
of memory installed in your machine?
16
votes
2answers
347 views
Why does ImageData need four times more memory?
Background
We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9. Many of our algorithms only work with an array representation of the images, ...
4
votes
0answers
122 views
How do I view the value of a variable in its raw, binary form, as stored in memory? [closed]
If I have a variable, how do I turn it into a list of bytes or a string containing its value as it's stored internally? To better illustrate what I mean, what I want to do could be done in C with ...
3
votes
1answer
125 views
Why does MemoryConstrained not abort when memory limit is exceeded?
I thought I had an idea that might lead to an answer to @yohbs' question, but as I was trying to check out the usage of MemoryConstrained[expr,b,failexpr], I ...
4
votes
3answers
186 views
How do I find the amount of free space on disk?
I know a workaround, where one calculates sizes of all files and then you subtract this from your default space, but is there any direct way to do it?
Simplicity and speed are important.
There's the ...
8
votes
2answers
480 views
How do you deal with very large datasets in Mathematica?
This question is in two related parts. The first is about dataset size, the second data wrangling.
Part 1:
I've built a naive Bayesian classifier on Mathematica. It uses ...
4
votes
1answer
142 views
Forget past outputs to free memory
While tinkering with a notebook that loads a large chunk of data in memory, I noticed that if I run it multiple times, the past chunks reside in memory. I understand that the reason (or one of) for ...
3
votes
1answer
108 views
How to efficiently take complement of two big lists?
I have a giant list l1 and some smaller lists l21, l21, ..., ...
7
votes
2answers
357 views
1
vote
1answer
287 views
NDSolve runs out of memory
I need to solve a second order ODE numerically. The ODE depends on two parameters (a,b). Things work fine when 'a' is small, but for large 'a' the solutions are oscillating rapidly and Mathematica ...
23
votes
4answers
855 views
What is the fastest way to count square-free words?
Background
A word is a string of letters in an alphabet. A square-free word has no adjacent repeating substring. For example, (in the ternary alphabet of {0,1,2}) the words 00, 012121, and 0212012021 ...
7
votes
2answers
165 views
Clean up list of partial outputs for memory
Context
I run long Mathematica sessions; sometimes I have Output up to Out[3000+]. Hence
I would like to be able to clear out some previous outputs which do use lots of memory.
Attempt
I am aware ...
4
votes
2answers
246 views
Implementation of builtin function Fold
In the documentation for fold, it says
Fold[f,x,list] gives the last element of FoldList[f,x,list].
But this is surely not the way it is implemented, right? ...
1
vote
1answer
126 views
Using block or module to free cache
I'm writing some code at the moment that schematically looks like:
(set initial conditions for some differential equation for given parameters (M,R,...))
...
15
votes
4answers
781 views
Plotting large datasets
I have a lot of data (e.g., 64x8192), to plot and it consumes too much memory (more than 2 GB) causing Mathematica to return "no memory available".
For example,
...
11
votes
1answer
316 views
Space-efficient null space of sparse array
I have a roughly 100,000 × 3,000 matrix (as a SparseArray) that I'd like to find the kernel (null space) of.
It has about 500,000 nonzero entries, all -1 or 1.
...
21
votes
1answer
2k views
Finding long strings of identical digits in transcendental numbers
Introduction
Describing the three main streams of present-day mathematical philosophy (formalism, Platonism and intuitionism) in a well-known book, The Emperor's New Mind, R. Penrose says:
...it ...
10
votes
2answers
220 views
8
votes
1answer
151 views
Reducing the memory footprint of lists of machine numbers
I have a large 1D dataset of double precision numbers that are stored in many external database files. In order to load the whole dataset into Mathematica 8 I iterate over the files ...
18
votes
2answers
756 views
Debugging memory leaks
I've written my incredibly complex, incredibly elegant analysis function, that works great on small test data. But when I run it on my real (bigger) data set it keeps running out of memory. It turns ...
21
votes
3answers
617 views
How to place an image inside a notebook, with the minimum memory footprint?
When creating a notebook with images, like photos, the file tends to quickly get big, which makes it difficult to use the notebook format for training material containing a lot of images.
Are there ...
0
votes
0answers
81 views
Running out of memory more gracefully [duplicate]
Possible Duplicate:
Aborting evaluation when the memory exceeds a certain limit
My script ran into the following error, with no further comments from the kernel.
No more memory ...
13
votes
3answers
545 views
Does parallel programming use up large quantities of memory in Mathematica?
When I start Mathematica with a fresh kernel and load a program I am developing to analyze biology data, the amount of used system memory is 1.9 GB (free memory is 14.5 GB). These memory values are ...
7
votes
1answer
323 views
Memory not freed after running Clear when using a Table
When I'm creating large lists with Table I see the memory footprint increasing.
...
8
votes
1answer
375 views
GeoDirection and GeoDistance Memory Leaks: How to Recover the Memory?
The functions GeoDirection and GeoDistance have memory leaks. To see this, run the example functions below and watch as the Physical Memory Available in the Windows Task Manager, Performance tab, ...
14
votes
3answers
357 views
Memory Leak in Frontend - anyone know a workaround?
When using Rasterize on a Graphics expression composed of many differently coloured Graphics objects memory is allocated in the frontend which is then not released or reused when a similar graphic ...
10
votes
1answer
144 views
Old values are not freed/garbage collected when you re-evaluate an assignment
For this code:
(* Cell 1 *)
generate := Module[{x}, x = Range[100 * 1000 * 1000]; x];
(* Cell 2 *)
g = generate[];
MemoryInUse[]
If I evaluate cell 2 ...
9
votes
2answers
388 views
Avoiding an unresponsive user interface in OS X
I have found that despite Mathematica's numerous updates, each of which have added much functionality, one fundamental issue remains unaddressed: The unresponsiveness of the UI when I make a mistake ...
22
votes
4answers
741 views
Aborting evaluation when the memory exceeds a certain limit
Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent ...
7
votes
2answers
392 views
Working with large data, pass-by-reference, compiled functions
I think I know the answer to this, but after two days of Googling, this is my last effort before going down an unwanted path...
I have a function that takes lots of large data as arguments as well as ...
23
votes
4answers
595 views
How to find out the (current) free memory on the system?
Is there any way in Mathematica to find out the currently free memory on the system Mathematica runs on (like the utility free shows on the command line)? I've ...
