Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.

learn more… | top users | synonyms

21
votes
1answer
393 views
+100

Efficient lazy weak compositions

In Mathematica all weak k-compositions of integer n can be generated using permutations of integer partitions: ...
10
votes
1answer
59 views

How to speed up the evaluation of a function with a large LeafCount but many repeated sub-functions for use in NIntegrate

General Context Apparently, the memory leak in NIntegrate is unavoidable pre-MMA-10.2. I have some (two-dimensional) numerical integration to do on a complicated ...
9
votes
0answers
113 views

Memory usage of compiled function: C vs WVM

Why does generating an array in "C"-compiled function uses 50% more memory than doing same thing in "WVM"-compiled function? ...
10
votes
1answer
390 views

Nintegrate has memory leak

Bug introduced in 6.0 and fixed in 10.2.0 NIntegrates uses memory and does not release it. For long loops with NIntegrate ...
0
votes
1answer
43 views

NIntegrate and memory use [duplicate]

I wonder why NIntegrate uses memory. I have a long integral computation repeated thousands times. The memory use then goes up and speed goes down. I think I spot a ...
1
vote
0answers
63 views

Is there a way to specify which kernel is used to import files (SOLVED)?

Following up on a previous question in which I encountered a memory leak in the JSON importer I have created a SandBox script that will run the Import and analysis on a separate kernel, which be ...
4
votes
1answer
68 views

Is there a way to monitor the memory used by mathematica as a dynamic expression?

I always use the terminal's htop for this, but wouldn't it be nice to visualize per kernel the memory used in a dynamic expression?
6
votes
2answers
104 views

Possible memory leak when importing JSON

Bug introduced in 10.0 and persisting though 10.2.0 This problem that I would like to resolve is how to free memory used Import a file when the file is no longer in use. I have a project in which ...
3
votes
0answers
62 views

Memory Management for Large Datasets [closed]

I've written some Mathematica code to generate polynomial roots. The code take an argument n as the highest degree of polynomial to solve for and then exports a file containing a list of the roots: ...
1
vote
0answers
31 views

Imported variables (.mx) are emptied occassionally (Linux, v10.0.2)

I have a severe problem with variables being "emptied" occassionally. I am not able to reproduce the behavior but this is basically what happens: 1) store list ...
3
votes
1answer
86 views

Simplify a huge expression with limited memory

I would like to perform some analytical sums such as the following ...
5
votes
1answer
103 views

memory leaks using NIntegrate on parallel kernel

Bug fixed in version 10.2.0 I'm running a large computation in parallel and the memory usage of the parallel kernels is increasing with every iteration of the calculation. Neither ...
5
votes
3answers
248 views

Huge matrices crashes in Mathematica

I have a matrix of data ~(70,000x70,000). I am searching connection between these data points. I use euclidean distance for that purpose. If it is less than my threshold (that means they have ...
3
votes
2answers
82 views

Add Results of ParallelMap in Parallel

I'm new to coding in parallel, but I really could use it. I need a command to call a function on each element in a huge list, and then add them all up. Currently I use something like this: ...
2
votes
0answers
63 views

MatrixRank of a large but sparse matrix

I want to find the rank of a very sparse, quite rectangular matrix mat, but I'm running out of RAM (I have 16 GB) if I try to use ...
0
votes
0answers
35 views

Is it possible to change default $HistoryLength [duplicate]

By default $HistoryLength is set to \[Infinity]. This is often unnecessary and can cause unecessary memory usage. I can of ...
1
vote
0answers
50 views

Does Sum calculate all terms before summing them?

After calculating the sum of a large number of large objects, using B = Sum[RandomInteger[{0, 1}, {10^6}], {100}]; I find that ...
0
votes
2answers
40 views

Notebook cells space on disk profiler

I have one single notebook with more than 500 mb... To control its growth, I would like to have a profiler of used space (not the live memory, but the space on disk), in such a way that I can easily ...
6
votes
0answers
143 views

NMinimize with NIntegrate (crash in symbolic evaluation, memory leak)

This is a "common" problem from what I've seen, but with a different spin. I have a function I use often that finds a fit of an expression to another expression with some free parameters (e.g. for ...
2
votes
0answers
71 views

Pros/Cons to updating in place

There are often times when I have to update a list. The following is an example: ...
2
votes
1answer
96 views

Memory usage optimization for a large Association/SparseArray

Reformulating my question in simpler terms. Take two lists list1 = RandomInteger[{1, 100000000}, 25000000]; list2 = RandomReal[{1, 10}, 25000000]; Make an ...
1
vote
0answers
78 views

Should I Remove the initial symbols of Module?

Often we use Module to construct complicate function. Consider that Module create new vars every time we run the function, should I remove the initial values at the end of Module-function?Or should I ...
0
votes
0answers
103 views

Out of Memory Error

I am currently constructing matrices, but I'm having problems with memory. When I evaluate some cells, the kernel quits due to lack of memory. I tried ...
0
votes
0answers
66 views

Handling really large Arrays (by possibly using lower precision) [duplicate]

I know there have been a few questions about efficient memory storage on mathematica.stackexchange, but I could not find a sufficient answer for my problem (except that it might not work at all). So ...
3
votes
2answers
110 views

memory management in Mathematica

Unlike using Unprotect[In,Out]; Clear[In,Out]; Protect[In,Out]; to clean the occupied memory of the whole notebook, I want to clean the memory dynamically and ...
1
vote
0answers
48 views

ImageAssemble and Export gif takes too much memory

Errr.... First Let's create some gifs: ...
1
vote
0answers
85 views

Out of memory problem on constructing large list in Mathematica running on Linux [closed]

I have Mathematica running on Mac as well as Linux, my Mac as 16GB of RAM with core i5 processor (2-core) and Linux has 24GB of RAM on i7 (4-core). The problem is that the following command: ...
0
votes
1answer
91 views

How do I manage my memory better when using the Map function?

So I'm attempting to write a Mathematica program that prints a range of natural numbers as the sum of four cubes. I wrote the following Mathematica code to try this, but am running into a memory ...
0
votes
0answers
132 views

Constructing and Reducing Tuples in a Memory Efficient Manner

This question is extension of a question I started here: Reduce the output from tuples by including symmetry? and will focus on higher order arrays and issues of optimization (similar to what I've ...
2
votes
0answers
71 views

Memory used by SparseArray objects

In dealing with a quantum mechanics problem, I am representing some many-body states in two different ways in Mathematica. For two parameters n and ...
1
vote
2answers
78 views

Table of ParametricPlots crashes Kernel

I can usually crash the Mathematica Kernel by using Table to generate a List of ...
5
votes
0answers
115 views

Memory leak when looping over FindInstance in version 10.0.0

Bug introduced in 10.0 and fixed in 10.1 When I run the following (simplified) code with FindInstance, my MemoryInUse is ...
1
vote
0answers
120 views

SystemException[MemoryAllocationFailure] with Memoized function [closed]

Version 10.1 Win7x64 Enterprise. This is more of a report than a question but others may have seen the same / have insights. I have some code (that I can't really post as it's too big) with a large ...
12
votes
1answer
175 views

ListConvolve Memory Hemorrhage

Bug introduced in 10.0.0 and fixed in 10.2.0. I wrote a routine that convolves an impulse response array with a test array and checks for the RMS difference between that convolution and a data ...
2
votes
4answers
118 views

calculating subsets of a matrix (L=40X40)

I need to calculate all $k$-possible subsets$(k:1,L)$ of matrix with size $L$. I am using subset function as following, ...
4
votes
1answer
167 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...
4
votes
1answer
130 views

Memory leak in MemberQ

Bug introduced in 10.0.0 and fixed in 10.0.2 Consider the following code. ...
0
votes
1answer
75 views

What is stored in the kernel's memory

I disabled "Show Suggestions Bar after last output" in the preferences. I begin my code with $HistoryLength = 0. After running my code, I run the following. ...
7
votes
2answers
165 views

Comparing memory consumption of two pieces of code

I have implemented something I want to do in two different ways, because I'm not sure which of the two approaches give best speed/memory efficiency. Comparing the speed of the two approaches is ...
3
votes
2answers
105 views

Memory Problem with Modules and NDSolve

I'm using NDSolve inside a module, and I appear to have a memory leak. The relevant code is: ...
-1
votes
2answers
158 views

Want to compute the permutations of {1, 2, …, 11} with only 3 GB of memory [duplicate]

There is another way to calculate Permutations [{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}] without triggering an error, I have 3 GB of RAM with WIN 7 Edit: This ...
2
votes
0answers
48 views

Does Mathematica support out-of-core Associations?

I often find myself collecting data in an Association[] which outgrows my computer's main memory. Does Mathematica support any out-of-core equivalent? Ideally, I ...
0
votes
0answers
181 views

Is there a way to give Mathematica extra memory?

I am running Mathematica 10 (Student Edition) on Mac OS X 10.7.5 on an old Mac Pro. I have lots of extra RAM that the system isn't touching, and I would like to give some of it to Mathematica in the ...
8
votes
0answers
110 views

What is stored in FrontEnd memory?

I quit the kernel, and closed all notebooks, leaving only the startup window. Yet when I open a new notebook, MemoryInUse[$FrontEnd] reveals that the FrontEnd still ...
1
vote
0answers
84 views

Avoid copying memory for each subkernel

Here is a simplified example illustrating my issue; ...
2
votes
1answer
100 views

Simple-looking Minimize takes for ever

Consider the following: Minimize[{(ξ - x)^2 + (η - y)^2, a ξ + b η == c}, {ξ, η}, Reals] After asking Mathematica to do this yesterday afternoon and forgetting ...
10
votes
0answers
152 views

How can the Theil-Sen estimator be made to work on larger datasets?

The Theil-Sen estimator finds the slope and intercept of a line passing through a set of points by calculating the median slope and median intercept of the set of lines passing through all possible ...
4
votes
1answer
92 views

Is there a way to constrain the memory used for a notebook (session)?

I know that we can constrain the memory usage of certain computation using MemoryConstrained. Is there a way to constrain the memory of each cell evaluation during ...
5
votes
1answer
258 views

Largest matrix size supported by Mathematica 10

I have need to store large square matrices in memory, some times sparse, some times not. I have used up to $2^{15}$ by $2^{15}$ as my upper bound so far, and Mathematica has handled it even if it took ...
7
votes
2answers
115 views

Compress uses too much memory and sometimes crashes the Kernel

Using Compress is a good idea to save memory when working with large expressions. The problem is that Compressing large ...