Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.

learn more… | top users | synonyms

18
votes
1answer
298 views

Efficient lazy weak compositions

In Mathematica all weak k-compositions of integer n can be generated using permutations of integer partitions: ...
10
votes
1answer
56 views

How to speed up the evaluation of a function with a large LeafCount but many repeated sub-functions for use in NIntegrate

General Context Apparently, the memory leak in NIntegrate is unavoidable pre-MMA-10.2. I have some (two-dimensional) numerical integration to do on a complicated ...
9
votes
0answers
111 views

Memory usage of compiled function: C vs WVM

Why does generating an array in "C"-compiled function uses 50% more memory than doing same thing in "WVM"-compiled function? ...
9
votes
1answer
385 views

Nintegrate has memory leak

Bug introduced in 6.0 and fixed in 10.2.0 NIntegrates uses memory and does not release it. For long loops with NIntegrate ...
0
votes
1answer
42 views

NIntegrate and memory use [duplicate]

I wonder why NIntegrate uses memory. I have a long integral computation repeated thousands times. The memory use then goes up and speed goes down. I think I spot a ...
1
vote
0answers
63 views

Is there a way to specify which kernel is used to import files (SOLVED)?

Following up on a previous question in which I encountered a memory leak in the JSON importer I have created a SandBox script that will run the Import and analysis on a separate kernel, which be ...
4
votes
1answer
68 views

Is there a way to monitor the memory used by mathematica as a dynamic expression?

I always use the terminal's htop for this, but wouldn't it be nice to visualize per kernel the memory used in a dynamic expression?
6
votes
2answers
101 views

Possible memory leak when importing JSON

Bug introduced in 10.0 and persisting though 10.2.0 This problem that I would like to resolve is how to free memory used Import a file when the file is no longer in use. I have a project in which ...
3
votes
0answers
62 views

Memory Management for Large Datasets [closed]

I've written some Mathematica code to generate polynomial roots. The code take an argument n as the highest degree of polynomial to solve for and then exports a file containing a list of the roots: ...
1
vote
0answers
30 views

Imported variables (.mx) are emptied occassionally (Linux, v10.0.2)

I have a severe problem with variables being "emptied" occassionally. I am not able to reproduce the behavior but this is basically what happens: 1) store list ...
3
votes
1answer
85 views

Simplify a huge expression with limited memory

I would like to perform some analytical sums such as the following ...
5
votes
1answer
103 views

memory leaks using NIntegrate on parallel kernel

Bug fixed in version 10.2.0 I'm running a large computation in parallel and the memory usage of the parallel kernels is increasing with every iteration of the calculation. Neither ...
5
votes
3answers
247 views

Huge matrices crashes in Mathematica

I have a matrix of data ~(70,000x70,000). I am searching connection between these data points. I use euclidean distance for that purpose. If it is less than my threshold (that means they have ...
3
votes
2answers
82 views

Add Results of ParallelMap in Parallel

I'm new to coding in parallel, but I really could use it. I need a command to call a function on each element in a huge list, and then add them all up. Currently I use something like this: ...
2
votes
0answers
62 views

MatrixRank of a large but sparse matrix

I want to find the rank of a very sparse, quite rectangular matrix mat, but I'm running out of RAM (I have 16 GB) if I try to use ...
0
votes
0answers
35 views

Is it possible to change default $HistoryLength [duplicate]

By default $HistoryLength is set to \[Infinity]. This is often unnecessary and can cause unecessary memory usage. I can of ...
1
vote
0answers
48 views

Does Sum calculate all terms before summing them?

After calculating the sum of a large number of large objects, using B = Sum[RandomInteger[{0, 1}, {10^6}], {100}]; I find that ...
0
votes
2answers
40 views

Notebook cells space on disk profiler

I have one single notebook with more than 500 mb... To control its growth, I would like to have a profiler of used space (not the live memory, but the space on disk), in such a way that I can easily ...
6
votes
0answers
140 views

NMinimize with NIntegrate (crash in symbolic evaluation, memory leak)

This is a "common" problem from what I've seen, but with a different spin. I have a function I use often that finds a fit of an expression to another expression with some free parameters (e.g. for ...
2
votes
0answers
71 views

Pros/Cons to updating in place

There are often times when I have to update a list. The following is an example: ...
2
votes
1answer
96 views

Memory usage optimization for a large Association/SparseArray

Reformulating my question in simpler terms. Take two lists list1 = RandomInteger[{1, 100000000}, 25000000]; list2 = RandomReal[{1, 10}, 25000000]; Make an ...
1
vote
0answers
78 views

Should I Remove the initial symbols of Module?

Often we use Module to construct complicate function. Consider that Module create new vars every time we run the function, should I remove the initial values at the end of Module-function?Or should I ...
0
votes
0answers
103 views

Out of Memory Error

I am currently constructing matrices, but I'm having problems with memory. When I evaluate some cells, the kernel quits due to lack of memory. I tried ...
0
votes
0answers
66 views

Handling really large Arrays (by possibly using lower precision) [duplicate]

I know there have been a few questions about efficient memory storage on mathematica.stackexchange, but I could not find a sufficient answer for my problem (except that it might not work at all). So ...
3
votes
2answers
110 views

memory management in Mathematica

Unlike using Unprotect[In,Out]; Clear[In,Out]; Protect[In,Out]; to clean the occupied memory of the whole notebook, I want to clean the memory dynamically and ...
1
vote
0answers
48 views

ImageAssemble and Export gif takes too much memory

Errr.... First Let's create some gifs: ...
1
vote
0answers
85 views

Out of memory problem on constructing large list in Mathematica running on Linux [closed]

I have Mathematica running on Mac as well as Linux, my Mac as 16GB of RAM with core i5 processor (2-core) and Linux has 24GB of RAM on i7 (4-core). The problem is that the following command: ...
0
votes
1answer
91 views

How do I manage my memory better when using the Map function?

So I'm attempting to write a Mathematica program that prints a range of natural numbers as the sum of four cubes. I wrote the following Mathematica code to try this, but am running into a memory ...
0
votes
0answers
131 views

Constructing and Reducing Tuples in a Memory Efficient Manner

This question is extension of a question I started here: Reduce the output from tuples by including symmetry? and will focus on higher order arrays and issues of optimization (similar to what I've ...
2
votes
0answers
71 views

Memory used by SparseArray objects

In dealing with a quantum mechanics problem, I am representing some many-body states in two different ways in Mathematica. For two parameters n and ...
1
vote
2answers
78 views

Table of ParametricPlots crashes Kernel

I can usually crash the Mathematica Kernel by using Table to generate a List of ...
5
votes
0answers
115 views

Memory leak when looping over FindInstance in version 10.0.0

Bug introduced in 10.0 and fixed in 10.1 When I run the following (simplified) code with FindInstance, my MemoryInUse is ...
1
vote
0answers
118 views

SystemException[MemoryAllocationFailure] with Memoized function [closed]

Version 10.1 Win7x64 Enterprise. This is more of a report than a question but others may have seen the same / have insights. I have some code (that I can't really post as it's too big) with a large ...
12
votes
1answer
175 views

ListConvolve Memory Hemorrhage

Bug introduced in 10.0.0 and fixed in 10.2.0. I wrote a routine that convolves an impulse response array with a test array and checks for the RMS difference between that convolution and a data ...
2
votes
4answers
118 views

calculating subsets of a matrix (L=40X40)

I need to calculate all $k$-possible subsets$(k:1,L)$ of matrix with size $L$. I am using subset function as following, ...
4
votes
1answer
166 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...
4
votes
1answer
130 views

Memory leak in MemberQ

Bug introduced in 10.0.0 and fixed in 10.0.2 Consider the following code. ...
0
votes
1answer
75 views

What is stored in the kernel's memory

I disabled "Show Suggestions Bar after last output" in the preferences. I begin my code with $HistoryLength = 0. After running my code, I run the following. ...
7
votes
2answers
161 views

Comparing memory consumption of two pieces of code

I have implemented something I want to do in two different ways, because I'm not sure which of the two approaches give best speed/memory efficiency. Comparing the speed of the two approaches is ...
3
votes
2answers
105 views

Memory Problem with Modules and NDSolve

I'm using NDSolve inside a module, and I appear to have a memory leak. The relevant code is: ...
-1
votes
2answers
158 views

Want to compute the permutations of {1, 2, …, 11} with only 3 GB of memory [duplicate]

There is another way to calculate Permutations [{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}] without triggering an error, I have 3 GB of RAM with WIN 7 Edit: This ...
2
votes
0answers
48 views

Does Mathematica support out-of-core Associations?

I often find myself collecting data in an Association[] which outgrows my computer's main memory. Does Mathematica support any out-of-core equivalent? Ideally, I ...
0
votes
0answers
179 views

Is there a way to give Mathematica extra memory?

I am running Mathematica 10 (Student Edition) on Mac OS X 10.7.5 on an old Mac Pro. I have lots of extra RAM that the system isn't touching, and I would like to give some of it to Mathematica in the ...
8
votes
0answers
110 views

What is stored in FrontEnd memory?

I quit the kernel, and closed all notebooks, leaving only the startup window. Yet when I open a new notebook, MemoryInUse[$FrontEnd] reveals that the FrontEnd still ...
1
vote
0answers
84 views

Avoid copying memory for each subkernel

Here is a simplified example illustrating my issue; ...
2
votes
1answer
99 views

Simple-looking Minimize takes for ever

Consider the following: Minimize[{(ξ - x)^2 + (η - y)^2, a ξ + b η == c}, {ξ, η}, Reals] After asking Mathematica to do this yesterday afternoon and forgetting ...
10
votes
0answers
152 views

How can the Theil-Sen estimator be made to work on larger datasets?

The Theil-Sen estimator finds the slope and intercept of a line passing through a set of points by calculating the median slope and median intercept of the set of lines passing through all possible ...
4
votes
1answer
92 views

Is there a way to constrain the memory used for a notebook (session)?

I know that we can constrain the memory usage of certain computation using MemoryConstrained. Is there a way to constrain the memory of each cell evaluation during ...
5
votes
1answer
257 views

Largest matrix size supported by Mathematica 10

I have need to store large square matrices in memory, some times sparse, some times not. I have used up to $2^{15}$ by $2^{15}$ as my upper bound so far, and Mathematica has handled it even if it took ...
7
votes
2answers
115 views

Compress uses too much memory and sometimes crashes the Kernel

Using Compress is a good idea to save memory when working with large expressions. The problem is that Compressing large ...