Questions about RAM usage, including optimizing memory use, avoiding and fixing leaks and releasing unused memory.

learn more… | top users | synonyms

20
votes
1answer
334 views
+100

Efficient lazy weak compositions

In Mathematica all weak k-compositions of integer n can be generated using permutations of integer partitions: ...
10
votes
1answer
57 views

How to speed up the evaluation of a function with a large LeafCount but many repeated sub-functions for use in NIntegrate

General Context Apparently, the memory leak in NIntegrate is unavoidable pre-MMA-10.2. I have some (two-dimensional) numerical integration to do on a complicated ...
11
votes
1answer
251 views

Where is my memory? — LibraryLink never returns the memory

Bug introduced in 8.0 or earlier and persisting through 10.2.0 This is an example LibaryLink code from Mathematica documentation, which takes an integer and return an integer list: ...
9
votes
0answers
112 views

Memory usage of compiled function: C vs WVM

Why does generating an array in "C"-compiled function uses 50% more memory than doing same thing in "WVM"-compiled function? ...
4
votes
1answer
167 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...
1
vote
0answers
78 views

Should I Remove the initial symbols of Module?

Often we use Module to construct complicate function. Consider that Module create new vars every time we run the function, should I remove the initial values at the end of Module-function?Or should I ...
3
votes
2answers
82 views

Add Results of ParallelMap in Parallel

I'm new to coding in parallel, but I really could use it. I need a command to call a function on each element in a huge list, and then add them all up. Currently I use something like this: ...
0
votes
1answer
43 views

NIntegrate and memory use [duplicate]

I wonder why NIntegrate uses memory. I have a long integral computation repeated thousands times. The memory use then goes up and speed goes down. I think I spot a ...
10
votes
1answer
388 views

Nintegrate has memory leak

Bug introduced in 6.0 and fixed in 10.2.0 NIntegrates uses memory and does not release it. For long loops with NIntegrate ...
0
votes
1answer
98 views

How to set all the numerical values to double

Is it possible to set the accuracy of every numeric value and the result of all computations to double like in Matlab? I want Mathematica to use just 3 or 4 digits ...
0
votes
1answer
100 views

FindMinimum runs out of Memory

I have a function which runs fine when I evaluate it. However, when I try to find its optimal parameters using FindMinimum it quickly consumes all available memory ...
1
vote
0answers
63 views

Is there a way to specify which kernel is used to import files (SOLVED)?

Following up on a previous question in which I encountered a memory leak in the JSON importer I have created a SandBox script that will run the Import and analysis on a separate kernel, which be ...
4
votes
1answer
68 views

Is there a way to monitor the memory used by mathematica as a dynamic expression?

I always use the terminal's htop for this, but wouldn't it be nice to visualize per kernel the memory used in a dynamic expression?
7
votes
2answers
163 views

Comparing memory consumption of two pieces of code

I have implemented something I want to do in two different ways, because I'm not sure which of the two approaches give best speed/memory efficiency. Comparing the speed of the two approaches is ...
22
votes
1answer
567 views

Excessive memory use by Mathematica kernel in 10.0.2 after executing simple commands

Bug introduced in 10.0 and fixed in 10.1 Warning: the code below may hang your system if you have the Suggestions Bar enabled in Mathematica < 10.1. When issuing the following simple ...
10
votes
1answer
496 views

GeoDirection and GeoDistance Memory Leaks: How to Recover the Memory?

Bug introduced in 7.0 and fixed in 9.0.0 The functions GeoDirection and GeoDistance have memory leaks. To see this, run the example functions below and watch as the Physical Memory Available in ...
6
votes
2answers
101 views

Possible memory leak when importing JSON

Bug introduced in 10.0 and persisting though 10.2.0 This problem that I would like to resolve is how to free memory used Import a file when the file is no longer in use. I have a project in which ...
5
votes
0answers
115 views

Memory leak when looping over FindInstance in version 10.0.0

Bug introduced in 10.0 and fixed in 10.1 When I run the following (simplified) code with FindInstance, my MemoryInUse is ...
12
votes
1answer
175 views

ListConvolve Memory Hemorrhage

Bug introduced in 10.0.0 and fixed in 10.2.0. I wrote a routine that convolves an impulse response array with a test array and checks for the RMS difference between that convolution and a data ...
3
votes
0answers
62 views

Memory Management for Large Datasets [closed]

I've written some Mathematica code to generate polynomial roots. The code take an argument n as the highest degree of polynomial to solve for and then exports a file containing a list of the roots: ...
1
vote
0answers
31 views

Imported variables (.mx) are emptied occassionally (Linux, v10.0.2)

I have a severe problem with variables being "emptied" occassionally. I am not able to reproduce the behavior but this is basically what happens: 1) store list ...
11
votes
2answers
779 views

Avoiding an unresponsive user interface in OS X

I have found that despite Mathematica's numerous updates, each of which have added much functionality, one fundamental issue remains unaddressed: The unresponsiveness of the UI when I make a mistake ...
5
votes
1answer
103 views

memory leaks using NIntegrate on parallel kernel

Bug fixed in version 10.2.0 I'm running a large computation in parallel and the memory usage of the parallel kernels is increasing with every iteration of the calculation. Neither ...
3
votes
1answer
86 views

Simplify a huge expression with limited memory

I would like to perform some analytical sums such as the following ...
5
votes
3answers
248 views

Huge matrices crashes in Mathematica

I have a matrix of data ~(70,000x70,000). I am searching connection between these data points. I use euclidean distance for that purpose. If it is less than my threshold (that means they have ...
2
votes
0answers
63 views

MatrixRank of a large but sparse matrix

I want to find the rank of a very sparse, quite rectangular matrix mat, but I'm running out of RAM (I have 16 GB) if I try to use ...
6
votes
0answers
141 views

NMinimize with NIntegrate (crash in symbolic evaluation, memory leak)

This is a "common" problem from what I've seen, but with a different spin. I have a function I use often that finds a fit of an expression to another expression with some free parameters (e.g. for ...
0
votes
0answers
35 views

Is it possible to change default $HistoryLength [duplicate]

By default $HistoryLength is set to \[Infinity]. This is often unnecessary and can cause unecessary memory usage. I can of ...
1
vote
0answers
48 views

Does Sum calculate all terms before summing them?

After calculating the sum of a large number of large objects, using B = Sum[RandomInteger[{0, 1}, {10^6}], {100}]; I find that ...
0
votes
2answers
40 views

Notebook cells space on disk profiler

I have one single notebook with more than 500 mb... To control its growth, I would like to have a profiler of used space (not the live memory, but the space on disk), in such a way that I can easily ...
2
votes
0answers
71 views

Pros/Cons to updating in place

There are often times when I have to update a list. The following is an example: ...
2
votes
1answer
96 views

Memory usage optimization for a large Association/SparseArray

Reformulating my question in simpler terms. Take two lists list1 = RandomInteger[{1, 100000000}, 25000000]; list2 = RandomReal[{1, 10}, 25000000]; Make an ...
2
votes
1answer
222 views

CUDALink ran out of available memory

I have a problem with automatic GPU memory management when using CUDA. The situation is the following: I create a simple CUDA function that operates on a 2D array (image), which does not take much ...
0
votes
0answers
103 views

Out of Memory Error

I am currently constructing matrices, but I'm having problems with memory. When I evaluate some cells, the kernel quits due to lack of memory. I tried ...
0
votes
0answers
66 views

Handling really large Arrays (by possibly using lower precision) [duplicate]

I know there have been a few questions about efficient memory storage on mathematica.stackexchange, but I could not find a sufficient answer for my problem (except that it might not work at all). So ...
8
votes
2answers
245 views

How can I tell how much memory is taken by an expression?

ByteCount and LeafCount don't really work, as shown below. Mathematica 9.0 on a 32-bit Windows machine: ...
7
votes
2answers
115 views

Compress uses too much memory and sometimes crashes the Kernel

Using Compress is a good idea to save memory when working with large expressions. The problem is that Compressing large ...
3
votes
2answers
110 views

memory management in Mathematica

Unlike using Unprotect[In,Out]; Clear[In,Out]; Protect[In,Out]; to clean the occupied memory of the whole notebook, I want to clean the memory dynamically and ...
1
vote
0answers
48 views
15
votes
2answers
2k views

How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
1
vote
2answers
78 views

Table of ParametricPlots crashes Kernel

I can usually crash the Mathematica Kernel by using Table to generate a List of ...
0
votes
1answer
91 views

How do I manage my memory better when using the Map function?

So I'm attempting to write a Mathematica program that prints a range of natural numbers as the sum of four cubes. I wrote the following Mathematica code to try this, but am running into a memory ...
1
vote
0answers
85 views

Out of memory problem on constructing large list in Mathematica running on Linux [closed]

I have Mathematica running on Mac as well as Linux, my Mac as 16GB of RAM with core i5 processor (2-core) and Linux has 24GB of RAM on i7 (4-core). The problem is that the following command: ...
0
votes
0answers
131 views

Constructing and Reducing Tuples in a Memory Efficient Manner

This question is extension of a question I started here: Reduce the output from tuples by including symmetry? and will focus on higher order arrays and issues of optimization (similar to what I've ...
2
votes
0answers
71 views

Memory used by SparseArray objects

In dealing with a quantum mechanics problem, I am representing some many-body states in two different ways in Mathematica. For two parameters n and ...
22
votes
3answers
845 views

DeleteDuplicatesBy is not performing as I'd hoped. Am I missing something?

Déjà vu: a new-in-v10 function should provide a better solution to an old problem, but my enthusiasm is curbed when I run timings. This time the function is ...
0
votes
0answers
116 views

What is the largest system of PDE you have solved with Mathematica?

I need to solve a system of partial differential equations. In the simplest case it consists of 2000 of PDE equations. In the most complex situation it consists up to 1 million of such equations. ...
2
votes
4answers
118 views

calculating subsets of a matrix (L=40X40)

I need to calculate all $k$-possible subsets$(k:1,L)$ of matrix with size $L$. I am using subset function as following, ...
1
vote
0answers
119 views

SystemException[MemoryAllocationFailure] with Memoized function [closed]

Version 10.1 Win7x64 Enterprise. This is more of a report than a question but others may have seen the same / have insights. I have some code (that I can't really post as it's too big) with a large ...
4
votes
1answer
130 views

Memory leak in MemberQ

Bug introduced in 10.0.0 and fixed in 10.0.2 Consider the following code. ...