Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.

learn more… | top users | synonyms

1
vote
0answers
69 views

Possible memory leak in FindMinimum? (MMA v10.0)

So I was running some code when I noticed Mathematica was gulping memory, and after some debugging I think the culprit is found. It turned out the function ...
2
votes
2answers
231 views

Follow each pixel brightness in set of images (Memory constrained)

I have a set of 2700 consecutive png images (8bit, 2048*2048 pixels). I would like to visualize the intensity variation of each pixel vs. image number. Here is an example of 10 images: ...
1
vote
1answer
91 views

NDSolve memory usage

I am trying to solve numerically a system of linear ODEs with some quickly varying driving functions. The basic command is: ...
0
votes
0answers
49 views

frequent slow evaluations for trivial expressions, large memory loaded

After loading a lot of data into memory and evaluating several hundred expressions without any problems, trivial expressions such as (1+2) take about 5-10 seconds to evaluate. Sometimes they are ...
36
votes
9answers
2k views

What would you ask Mathematica to do on a big system?

Our hardware supplier has given us the opportunity to test Mathematica on a big Windows system. The specs will be 1 to 4 TB of working memory and 12 sockets with 18 cores each. I got 1 full week of ...
3
votes
2answers
82 views

`IntegerPartitions` without duplicates

I need to apply a function to integer partitions of many integers, but only the partitions without duplicate numbers. Select[IntegerPartitions[n], DuplicateFreeQ] ...
0
votes
0answers
29 views

Why does NDSolve and ParametricNDSolveValue show different memory manage in same loop?

This is a follow-up question about How to delete InterpolationFunction created by NDSolve/ParametricNDSolve?. In the previous question, J. M.♦ and Albert Retey showed $HistoryLength = 0 and Clear[] ...
0
votes
0answers
53 views

ButterworthFilterModel consumes the memory too much

I have a simple filter which is a result of the these questions here and here I have to use this filter in a Do loop but I found that this filter consume the ...
1
vote
0answers
54 views

How to delete InterpolationFunction created by NDSolve/ParametricNDSolve?

I have faced a uncommon problem. The situation is that I need use ParametricNDSolve/ParametricNDSolveValue to solve ODEs system in the non-linear model fitting process. So every iteration generating a ...
1
vote
0answers
72 views

Lower memory usage and speed nested loops

I'm trying to make some tracking with Mathematica. ...
1
vote
0answers
47 views

Running out of memory when doing non-commutative arithmetic

I need to calculate particular coefficients of the composition of two power series in non-commutative variables. The code below which works fine truncates power series in degree $n=5$. For $n=6$ (the ...
14
votes
3answers
246 views

On making a graphical real-time memory usage monitor

I'd like to make a graphical version of Dynamic[Refresh[MemoryInUse[], UpdateInterval -> 1]] A naive implementation would be something like: ...
1
vote
1answer
150 views

Simplify the huge determinant product

I try to get the following determinant product result, ...
23
votes
1answer
660 views

Efficient lazy weak compositions

In Mathematica all weak k-compositions of integer n can be generated using permutations of integer partitions: ...
10
votes
1answer
87 views

How to speed up the evaluation of a function with a large LeafCount but many repeated sub-functions for use in NIntegrate

General Context Apparently, the memory leak in NIntegrate is unavoidable pre-MMA-10.2. I have some (two-dimensional) numerical integration to do on a complicated ...
11
votes
0answers
192 views

Memory usage of compiled function: C vs WVM

Reported to WRI [CASE:3406803] Why does generating an array in "C"-compiled function uses 50% more memory than doing same thing in ...
11
votes
1answer
425 views

Nintegrate has memory leak

Bug introduced in 6.0 and fixed in 10.2.0 NIntegrates uses memory and does not release it. For long loops with NIntegrate ...
0
votes
1answer
62 views

NIntegrate and memory use [duplicate]

I wonder why NIntegrate uses memory. I have a long integral computation repeated thousands times. The memory use then goes up and speed goes down. I think I spot a ...
2
votes
0answers
76 views

Is there a way to specify which kernel is used to import files (SOLVED)?

Following up on a previous question in which I encountered a memory leak in the JSON importer I have created a SandBox script that will run the Import and analysis on a separate kernel, which be ...
5
votes
1answer
99 views

Is there a way to monitor the memory used by mathematica as a dynamic expression?

I always use the terminal's htop for this, but wouldn't it be nice to visualize per kernel the memory used in a dynamic expression?
6
votes
2answers
133 views

Possible memory leak when importing JSON

Bug introduced in 10.0 and persisting though 10.2.0 This problem that I would like to resolve is how to free memory used Import a file when the file is no longer in use. I have a project in which ...
3
votes
0answers
74 views

Memory Management for Large Datasets [closed]

I've written some Mathematica code to generate polynomial roots. The code take an argument n as the highest degree of polynomial to solve for and then exports a file containing a list of the roots: ...
1
vote
0answers
37 views

Imported variables (.mx) are emptied occassionally (Linux, v10.0.2)

I have a severe problem with variables being "emptied" occassionally. I am not able to reproduce the behavior but this is basically what happens: 1) store list ...
3
votes
1answer
107 views

Simplify a huge expression with limited memory

I would like to perform some analytical sums such as the following ...
5
votes
1answer
127 views

memory leaks using NIntegrate on parallel kernel

Bug fixed in version 10.2.0 I'm running a large computation in parallel and the memory usage of the parallel kernels is increasing with every iteration of the calculation. Neither ...
5
votes
3answers
270 views

Huge matrices crashes in Mathematica

I have a matrix of data ~(70,000x70,000). I am searching connection between these data points. I use euclidean distance for that purpose. If it is less than my threshold (that means they have ...
3
votes
2answers
101 views

Add Results of ParallelMap in Parallel

I'm new to coding in parallel, but I really could use it. I need a command to call a function on each element in a huge list, and then add them all up. Currently I use something like this: ...
2
votes
0answers
76 views

MatrixRank of a large but sparse matrix

I want to find the rank of a very sparse, quite rectangular matrix mat, but I'm running out of RAM (I have 16 GB) if I try to use ...
0
votes
0answers
47 views

Is it possible to change default $HistoryLength [duplicate]

By default $HistoryLength is set to \[Infinity]. This is often unnecessary and can cause unecessary memory usage. I can of ...
1
vote
0answers
57 views

Does Sum calculate all terms before summing them?

After calculating the sum of a large number of large objects, using B = Sum[RandomInteger[{0, 1}, {10^6}], {100}]; I find that ...
2
votes
2answers
73 views

Notebook cells space on disk profiler

I have one single notebook with more than 500 mb... To control its growth, I would like to have a profiler of used space (not the live memory, but the space on disk), in such a way that I can easily ...
7
votes
1answer
238 views

NMinimize with NIntegrate (crash in symbolic evaluation, memory leak)

This is a "common" problem from what I've seen, but with a different spin. I have a function I use often that finds a fit of an expression to another expression with some free parameters (e.g. for ...
2
votes
0answers
75 views

Pros/Cons to updating in place

There are often times when I have to update a list. The following is an example: ...
2
votes
1answer
138 views

Memory usage optimization for a large Association/SparseArray

Reformulating my question in simpler terms. Take two lists list1 = RandomInteger[{1, 100000000}, 25000000]; list2 = RandomReal[{1, 10}, 25000000]; Make an ...
1
vote
0answers
80 views

Should I Remove the initial symbols of Module?

Often we use Module to construct complicate function. Consider that Module create new vars every time we run the function, should I remove the initial values at the end of Module-function?Or should I ...
0
votes
0answers
108 views

Out of Memory Error

I am currently constructing matrices, but I'm having problems with memory. When I evaluate some cells, the kernel quits due to lack of memory. I tried ...
0
votes
1answer
146 views

Which platform is ideal for performance?

I have been using NDSolve, and I noticed that when you decrease the MaxStepSize, possible singularities are resolved. If I ...
0
votes
0answers
67 views

Handling really large Arrays (by possibly using lower precision) [duplicate]

I know there have been a few questions about efficient memory storage on mathematica.stackexchange, but I could not find a sufficient answer for my problem (except that it might not work at all). So ...
3
votes
2answers
127 views

memory management in Mathematica

Unlike using Unprotect[In,Out]; Clear[In,Out]; Protect[In,Out]; to clean the occupied memory of the whole notebook, I want to clean the memory dynamically and ...
1
vote
0answers
52 views

ImageAssemble and Export gif takes too much memory

Errr.... First Let's create some gifs: ...
1
vote
0answers
166 views

Out of memory problem on constructing large list in Mathematica running on Linux [closed]

I have Mathematica running on Mac as well as Linux, my Mac as 16GB of RAM with core i5 processor (2-core) and Linux has 24GB of RAM on i7 (4-core). The problem is that the following command: ...
0
votes
1answer
116 views

How do I manage my memory better when using the Map function?

So I'm attempting to write a Mathematica program that prints a range of natural numbers as the sum of four cubes. I wrote the following Mathematica code to try this, but am running into a memory ...
0
votes
0answers
153 views

Constructing and Reducing Tuples in a Memory Efficient Manner

This question is extension of a question I started here: Reduce the output from tuples by including symmetry? and will focus on higher order arrays and issues of optimization (similar to what I've ...
2
votes
0answers
97 views

Memory used by SparseArray objects

In dealing with a quantum mechanics problem, I am representing some many-body states in two different ways in Mathematica. For two parameters n and ...
1
vote
2answers
81 views

Table of ParametricPlots crashes Kernel

I can usually crash the Mathematica Kernel by using Table to generate a List of ...
5
votes
0answers
126 views

Memory leak when looping over FindInstance in version 10.0.0

Bug introduced in 10.0 and fixed in 10.1 When I run the following (simplified) code with FindInstance, my MemoryInUse is ...
1
vote
0answers
248 views

SystemException[MemoryAllocationFailure] with Memoized function [closed]

Version 10.1 Win7x64 Enterprise. This is more of a report than a question but others may have seen the same / have insights. I have some code (that I can't really post as it's too big) with a large ...
12
votes
1answer
187 views

ListConvolve Memory Hemorrhage

Bug introduced in 10.0.0 and fixed in 10.2.0. I wrote a routine that convolves an impulse response array with a test array and checks for the RMS difference between that convolution and a data ...
2
votes
4answers
125 views

calculating subsets of a matrix (L=40X40)

I need to calculate all $k$-possible subsets$(k:1,L)$ of matrix with size $L$. I am using subset function as following, ...
4
votes
1answer
229 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...