Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.

learn more… | top users | synonyms

2
votes
0answers
219 views

Project Euler (memory and speed issues)

Let n be a positive integer. An integer triple (a, b, c) is called a factorisation triple of n if: 1 ≤ a ≤ b ≤ c a·b·c = n. Define f(n) to be a + b + c for the factorisation triple (a, b, c) of n ...
8
votes
1answer
631 views

Mathematica memory management for large arrays

I have come across a weird phenomenon in Mathematica when dealing with large arrays. When generating a list with all the possible subsets of three elements of another list (thus having elements which ...
3
votes
2answers
272 views

Storing result from nested Table in a File for further computations

Consider the following: Table[Table[x = 20; y = 30; 2 i x + 3 j y , {i, 10}], {j, 3}] Suppose I want the result of this nested Table to be stored to an ...
0
votes
1answer
226 views

Why does memory run out when adding same-sized matrices in a loop?

I have written a module that it generates a 101x101x51 zero matrix at first. At every iteration, it generates a list by funct1 and based on this list, if list is empty it gives the previous result. ...
17
votes
3answers
782 views

How to clear parts of a memoized function?

I have a function of two variables, e.g.: f[a_, b_] := f[a, b] = something f[a - 1, b - 1] etc With the above code I used the concept of memoization to speed up ...
14
votes
1answer
546 views

Efficient code for tallying entries in very large lists

I am dealing with data sets containing tens of millions of (hashable) entries and simply using the Tally function to count the frequency of each unique list element ...
5
votes
1answer
256 views

Processing and storing a graph with a large number of vertices s.t. k-hop neighborhoods are efficiently retrievable

I have an unweighted, undirected graph $G$ with $(v_1, ..., v_N) \in V$ vertices, where each vertex in the lattice represents a string of some length ($\leq 200$ characters), and I need to construct ...
0
votes
0answers
79 views

Is there a way to ask Mathematica for the RAM installed? [duplicate]

Possible Duplicate: How to find out the (current) free memory on the system? Do we kwow of a function to get the installed amount of physical system memory RAM of my Windows PC?
19
votes
2answers
745 views

Why does ImageData need four times more memory?

Background We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
4
votes
1answer
271 views

Why does MemoryConstrained not abort when memory limit is exceeded?

I thought I had an idea that might lead to an answer to @yohbs' question, but as I was trying to check out the usage of MemoryConstrained[expr,b,failexpr], I ...
5
votes
3answers
418 views

How do I find the amount of free space on disk?

I know a workaround, where one calculates sizes of all files and then you subtract this from your default space, but is there any direct way to do it? Simplicity and speed are important. There's the ...
12
votes
2answers
2k views

How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
4
votes
1answer
590 views

Forget past outputs to free memory

While tinkering with a notebook that loads a large chunk of data in memory, I noticed that if I run it multiple times, the past chunks reside in memory. I understand that the reason (or one of) for ...
3
votes
1answer
185 views

How to efficiently take complement of two big lists?

I have a giant list l1 and some smaller lists l21, l21, ..., ...
7
votes
2answers
918 views

Memory running out generating ContourPlot3D

The code below ...
2
votes
1answer
766 views

NDSolve runs out of memory

I need to solve a second order ODE numerically. The ODE depends on two parameters (a,b). Things work fine when 'a' is small, but for large 'a' the solutions are oscillating rapidly and Mathematica ...
23
votes
4answers
1k views

What is the fastest way to count square-free words?

Background A word is a string of letters in an alphabet. A square-free word has no adjacent repeating substring. For example, (in the ternary alphabet of {0,1,2}) the words 00, 012121, and 0212012021 ...
7
votes
2answers
251 views

Clean up list of partial outputs for memory

Context I run long Mathematica sessions; sometimes I have Output up to Out[3000+]. Hence I would like to be able to clear out some previous outputs which do use lots of memory. Attempt I am aware ...
4
votes
2answers
318 views

Implementation of builtin function Fold

In the documentation for fold, it says Fold[f,x,list] gives the last element of FoldList[f,x,list]. But this is surely not the way it is implemented, right? ...
1
vote
1answer
197 views

Using block or module to free cache

I'm writing some code at the moment that schematically looks like: (set initial conditions for some differential equation for given parameters (M,R,...)) ...
21
votes
4answers
2k views

Plotting large datasets

I have a lot of data (e.g., 64x8192), to plot and it consumes too much memory (more than 2 GB) causing Mathematica to return "no memory available". For example, ...
11
votes
1answer
558 views

Space-efficient null space of sparse array

I have a roughly 100,000 × 3,000 matrix (as a SparseArray) that I'd like to find the kernel (null space) of. It has about 500,000 nonzero entries, all -1 or 1. ...
25
votes
1answer
3k views

Finding long strings of identical digits in transcendental numbers

Introduction Describing the three main streams of present-day mathematical philosophy (formalism, Platonism and intuitionism) in a well-known book, The Emperor's New Mind, R. Penrose says: ...it ...
11
votes
2answers
335 views

Memory leak in FE?

A very abridged example of what was originally a major leak ...
8
votes
1answer
194 views

Reducing the memory footprint of lists of machine numbers

I have a large 1D dataset of double precision numbers that are stored in many external database files. In order to load the whole dataset into Mathematica 8 I iterate over the files ...
24
votes
2answers
2k views

Debugging memory leaks

I've written my incredibly complex, incredibly elegant analysis function, that works great on small test data. But when I run it on my real (bigger) data set it keeps running out of memory. It turns ...
22
votes
3answers
1k views

How to place an image inside a notebook, with the minimum memory footprint?

When creating a notebook with images, like photos, the file tends to quickly get big, which makes it difficult to use the notebook format for training material containing a lot of images. Are there ...
21
votes
3answers
1k views

Does parallel programming use up large quantities of memory in Mathematica?

When I start Mathematica with a fresh kernel and load a program I am developing to analyze biology data, the amount of used system memory is 1.9 GB (free memory is 14.5 GB). These memory values are ...
7
votes
1answer
741 views

Memory not freed after running Clear when using a Table

When I'm creating large lists with Table I see the memory footprint increasing. ...
9
votes
1answer
463 views

GeoDirection and GeoDistance Memory Leaks: How to Recover the Memory?

The functions GeoDirection and GeoDistance have memory leaks. To see this, run the example functions below and watch as the Physical Memory Available in the Windows Task Manager, Performance tab, ...
18
votes
3answers
554 views

Memory Leak in Frontend - anyone know a workaround?

When using Rasterize on a Graphics expression composed of many differently coloured Graphics objects memory is allocated in the frontend which is then not released or reused when a similar graphic ...
11
votes
1answer
274 views

Old values are not freed/garbage collected when you re-evaluate an assignment

For this code: (* Cell 1 *) generate := Module[{x}, x = Range[100 * 1000 * 1000]; x]; (* Cell 2 *) g = generate[]; MemoryInUse[] If I evaluate cell 2 ...
9
votes
2answers
710 views

Avoiding an unresponsive user interface in OS X

I have found that despite Mathematica's numerous updates, each of which have added much functionality, one fundamental issue remains unaddressed: The unresponsiveness of the UI when I make a mistake ...
30
votes
4answers
2k views

Aborting evaluation when the memory exceeds a certain limit

Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent ...
7
votes
2answers
704 views

Working with large data, pass-by-reference, compiled functions

I think I know the answer to this, but after two days of Googling, this is my last effort before going down an unwanted path... I have a function that takes lots of large data as arguments as well as ...
24
votes
4answers
1k views

How to find out the (current) free memory on the system?

Is there any way in Mathematica to find out the currently free memory on the system Mathematica runs on (like the utility free shows on the command line)? I've ...