Questions tagged [memory]
Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.
113
questions
38
votes
2
answers
5k
views
Debugging memory leaks
I've written my incredibly complex, incredibly elegant analysis function, that works great on small test data. But when I run it on my real (bigger) data set it keeps running out of memory. It turns ...
43
votes
5
answers
5k
views
Aborting evaluation when the memory exceeds a certain limit
Sometimes when generating a lot of data the memory usage shoots beyond its physical realm and my laptop freezes. There is no way to stop Mathematica when this happens. Task manager (or its equivalent ...
34
votes
3
answers
3k
views
How to place an image inside a notebook, with the minimum memory footprint?
When creating a notebook with images, like photos, the file tends to quickly get big, which makes it difficult to use the notebook format for training material containing a lot of images.
Are there ...
36
votes
3
answers
2k
views
DeleteDuplicatesBy is not performing as I'd hoped. Am I missing something?
Déjà vu: a new-in-v10 function should provide a better solution to an old problem, but my enthusiasm is curbed when I run timings. This time the function is ...
37
votes
3
answers
3k
views
How to clear parts of a memoized function?
I have a function of two variables, e.g.:
f[a_, b_] := f[a, b] = something f[a - 1, b - 1] etc
With the above code I used the concept of memoization to speed up ...
16
votes
1
answer
1k
views
Space-efficient null space of sparse array
I have a roughly 100,000 × 3,000 matrix (as a SparseArray) that I'd like to find the kernel (null space) of.
It has about 500,000 nonzero entries, all -1 or 1.
...
8
votes
2
answers
238
views
How to relate memory usage with occupied positions of SparseArrays?
What is the relation of memory usage of a SparseArray and the number of its occupied positions?
Let's say you build a $100000000 \times 10$ ...
33
votes
4
answers
5k
views
Plotting large datasets
I have a lot of data (e.g., 64x8192), to plot and it consumes too much memory (more than 2 GB) causing Mathematica to return "no memory available".
For example,
...
34
votes
6
answers
3k
views
How to find out the (current) free memory on the system?
Is there any way in Mathematica to find out the currently free memory on the system Mathematica runs on (like the utility free shows on the command line)? I've ...
22
votes
3
answers
923
views
Memory Leak in Frontend - anyone know a workaround?
When using Rasterize on a Graphics expression composed of many differently coloured Graphics objects memory is allocated in the frontend which is then not released or reused when a similar graphic ...
14
votes
1
answer
696
views
NIntegrate has a memory leak
Bug introduced in 6.0 and fixed in 10.2.0
NIntegrate uses memory and does not release it. For long loops with NIntegrate, ...
47
votes
3
answers
4k
views
Smooth Peter de Jong attractor
Today I was playing with Peter de Jong attractor.
At the bottom of the page I've linked there are beautiful examples like:
My attempts are not so great:
It is around 10^5 points. For more than 5*10^...
27
votes
1
answer
4k
views
Finding long strings of identical digits in transcendental numbers
Introduction
Describing the three main streams of present-day mathematical philosophy (formalism, Platonism and intuitionism) in a well-known book, The Emperor's New Mind, R. Penrose says:
...it ...
20
votes
2
answers
6k
views
How do you deal with very large datasets in Mathematica?
This question is in two related parts. The first is about dataset size, the second data wrangling.
Part 1:
I've built a naive Bayesian classifier on Mathematica. It uses ...
6
votes
3
answers
5k
views
How to remove all of the memory used by Mathematica
Backgroud
When I create the notebook,it will be made mess by me sometimes.If I use Remove to remove those variables,the memory cannot release still.So I have to ...
20
votes
1
answer
497
views
Memory usage of compiled function: C vs WVM
Reported to WRI [CASE:3406803]
Why does generating an array in "C"-compiled function uses 50% more memory than doing same thing in ...
12
votes
2
answers
1k
views
How to un-eat memory?
Consider this code:
...
11
votes
1
answer
437
views
Is it possible to lower the bitwidth of (floating point) numbers?
The answer to this question will most probably be a definite no. Nevertheless, I would like to know whether it is possible to represent reals in the memory on less than the standard 32 or 64 bits (...
9
votes
2
answers
450
views
Clean up list of partial outputs for memory
Context
I run long Mathematica sessions; sometimes I have Output up to Out[3000+]. Hence
I would like to be able to clear out some previous outputs which do use lots of memory.
Attempt
I am aware ...
34
votes
1
answer
1k
views
Efficient lazy weak compositions
In Mathematica all weak k-compositions of integer n can be generated using permutations of integer partitions:
...
26
votes
1
answer
2k
views
Excessive memory use by Mathematica kernel in 10.0.2 after executing simple commands
Bug introduced in 10.0 and fixed in 10.1
Warning: the code below may hang your system if you have the Suggestions Bar enabled in Mathematica < 10.1.
When issuing the following simple statements ...
17
votes
2
answers
3k
views
How to force Mathematica to clean up the cache
(see the bottom of this question for the MWE)
I'm working with a set of numerical data: numerous (a few thousands) files in the style of list of vectors (quadruplets x, y, vx, vy). Files are named as ...
16
votes
2
answers
985
views
Remove versus ClearAll
What is the difference between Remove and ClearAll?
I thought that "not being recognized by Mathematica", which is the effect ...
12
votes
2
answers
1k
views
Memory efficient generation and selection of tuples
The short of this question: I need to find a bunch of n-tuples of {0,1} which do not fail to satisfy a set of (non-linear polynomial) equations, without hogging up all of my memory by trying to ...
11
votes
2
answers
531
views
Building sparse array from packed arrays
Suppose I have two large packed arrays a and b, that respectively contain the position of the elements of a square matrix (i.e. <...
11
votes
3
answers
402
views
Can I return lists with different dimensions from a compiled function?
Sometimes I'll need several lists with different dimensions to be returned by a compiled function, but compiled function will fail if an irregular list contained:
...
9
votes
1
answer
1k
views
Memory not freed after running Clear when using a Table [duplicate]
When I'm creating large lists with Table I see the memory footprint increasing.
...
5
votes
1
answer
169
views
Memory consumption problem when reading png images with Image`ImportExportDump`ImageReadPNG
For importing png images I use often:
image = First@Image`ImportExportDump`ImageReadPNG[fileName]
(see: ?*`*PNG*)
since this is faster than:
...
5
votes
2
answers
666
views
Storing result from nested Table in a File for further computations
Consider the following:
Table[Table[x = 20; y = 30; 2 i x + 3 j y , {i, 10}], {j, 3}]
Suppose I want the result of this nested Table to be stored to an ...
4
votes
3
answers
354
views
Is there something wrong with this piece of code involving Manipulate?
I am playing around with the Manipulate sentence. I think the following code is right, and actually Mathematica 9.0 doesn't return any error message, but I am ...
36
votes
2
answers
1k
views
How do I disable the stack tracing feature in Mathematica 11?
Bug was introduced in 11.0 and persisting through 11.1
It looks like the Mathematica 11 stack tracing feature (see red ellipsis in front of any warning message) prevents the garbage collector from ...
21
votes
2
answers
1k
views
Why does ImageData need four times more memory?
Background
We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
20
votes
2
answers
1k
views
Module trash collection behaviour
I have asked this question before on the Wolfram community forum (http://community.wolfram.com/groups/-/m/t/83845?p_p_auth=Gxoxe65B) but not with great success.
Module creates temporary variables of ...
18
votes
1
answer
988
views
Efficient code for tallying entries in very large lists
I am dealing with data sets containing tens of millions of (hashable) entries and simply using the Tally function to count the frequency of each unique list element ...
14
votes
2
answers
611
views
Memory leak with NDSolve
Bug solved in 11.2
Bug introduced in 10.3 or earlier and persists through 11.0
Edit The technical team confirms the memory leak in NDSolve and has forwarded an ...
13
votes
2
answers
1k
views
Giving JLink Huge Memory by Default
I have noticed that Mathematica makes use of Java by way of JLink, for example, in the extensions such as DatabaseLink and (I think) RLink. It also seems like Java is a first-class citizen for writing ...
13
votes
1
answer
2k
views
Mathematica memory management for large arrays
I have come across a weird phenomenon in Mathematica when dealing with large arrays. When generating a list with all the possible subsets of three elements of another list (thus having elements which ...
12
votes
2
answers
1k
views
Why does this simple program leak memory?
I have a simple Mathematica program which writes some plots to image files for later conversion into a movie. Unfortunately, the program leaks so much memory that it quickly exhausts all 12G of RAM ...
12
votes
2
answers
541
views
Memory leak in FE?
A very abridged example of what was originally a major leak
...
11
votes
1
answer
350
views
Where is my memory? -- LibraryLink never returns the memory
Bug introduced in 8.0 or earlier and persisting through 10.3
This is an example LibaryLink code from Mathematica documentation, which takes an integer and return an integer list:
...
9
votes
2
answers
288
views
Comparing memory consumption of two pieces of code
I have implemented something I want to do in two different ways, because I'm not sure which of the two approaches give best speed/memory efficiency. Comparing the speed of the two approaches is ...
8
votes
3
answers
1k
views
How to fix this memory leak
I am using halirutan's code in this thread to unflatten a list. It works very well but the problem is that the Module's local function f is not deleted when the ...
8
votes
2
answers
1k
views
Working with large data, pass-by-reference, compiled functions
I think I know the answer to this, but after two days of Googling, this is my last effort before going down an unwanted path...
I have a function that takes lots of large data as arguments as well as ...
6
votes
3
answers
743
views
How do I find the amount of free space on disk?
I know a workaround, where one calculates sizes of all files and then you subtract this from your default space, but is there any direct way to do it?
Simplicity and speed are important.
There's the ...
6
votes
1
answer
194
views
Code in separate cells causes a kernel crash [duplicate]
Bug introduced in 11.2
If I put the following code in separate cells the Kernel crashes (dies). If I put them together in the same cell it works fine. Version 11.3.0 for Windows.
...
5
votes
1
answer
2k
views
Mathematica takes way too much memory?
Here is some simple code:
...
5
votes
4
answers
561
views
Memory Problem with Modules and NDSolve
I'm using NDSolve inside a module, and I appear to have a memory leak. The relevant code is:
...
25
votes
1
answer
2k
views
Generating and rendering extremely large graphs
I'm interested in generating and rendering extremely large graphs, i.e., ones having $10^6 \to 10^8$ vertices. (These graphs will ultimately be rendered in PDF or PNG and printed on large, poster-...
23
votes
1
answer
262
views
Possible Memory Leak Issues in NestWhile
Consider the following code snippet:
NestWhile[{#[[1]]+1,[email protected];ConstantArray[0,{100, 100}]}&,
{1}, (#[[1]] <= 10000) &, 2]
In theory, when ...
23
votes
3
answers
3k
views
Does parallel programming use up large quantities of memory in Mathematica?
When I start Mathematica with a fresh kernel and load a program I am developing to analyze biology data, the amount of used system memory is 1.9 GB (free memory is 14.5 GB). These memory values are ...