Questions on optimizing Mathematica code for higher performance. This may mean faster execution, lower memory usage or both. Not to be confused with mathematical optimization.
65
votes
1answer
2k views
File-backed lists/variables for handling large data
Background
Currently I am working with some large data (most of it generated by Mathematica itself). I usually find it a hassle to do this. For example, I just exported a large amount of data to ...
49
votes
8answers
11k views
How to make an inkblot?
How to effectively create a polygon that looks like a realistic inkblot? So far, I could come up with this (borrowing from Ed Pegg Jr.'s Rorschach demonstration):
...
44
votes
4answers
743 views
What best practices or performance considerations are there for choosing between Cases, Position, Pick and Select?
Cases, Select,Pick and Position each have different ...
42
votes
2answers
2k views
Shaving the last 50 ms off NMinimize
The background:
I have to fit an objective function for ~10 000 datasets in near real time (with near real time being on the order of about 10 seconds). I compile the huge objective function, so that ...
40
votes
6answers
4k views
33
votes
1answer
561 views
Profiling from Mathematica
I've always wished I could do some profiling like you get in Wolfram Workbench, but directly from Mathematica, without using or having Workbench.
How can I do it? How? It would be really nice!! ...
33
votes
2answers
847 views
Performance tuning for game solving (peg solitaire | senku)
There's a game I saw at a friend's yesterday, that I often see at people's homes, but never for enough time to think on it too hard. It's called peg solitaire (thanks @R.M).
So I came home and I ...
33
votes
1answer
880 views
Adaptive sampling for slow to compute functions in 2D
EDIT: Although I have posted an answer based on my current progress, this in incomplete. Please see the "open issues" section in the answer.
Most plotting functions in Mathematica adjust the ...
32
votes
2answers
9k views
How can I use Mathematica's graph functions to cheat at Boggle?
This question is inspired by a Stack Overflow question that I decided to solve using Mathematica. In addition to Mathematica, I thought I'd use some of the new version's graph-related functionality, ...
32
votes
2answers
980 views
Internal`Bag inside Compile
Since Internal`Bag, Internal`StuffBag and Internal`BagPart can be compiled down, it is a ...
31
votes
4answers
1k views
Speeding up this fractal-generating code
I used the code below (which is a sample from this gist containing more similar code) in my answer to my own question about Mandelbrot-like sets for functions other than the simple quadratic on ...
30
votes
9answers
1k views
Happy 2K prime question
This being the Q number 2K in the site, and this being the day we got the confirmation of mathematica.se graduating soon, I think a celebration question is in order.
So...
What is the fastest way to ...
29
votes
2answers
861 views
What is a Mathematica packed array?
A simple sounding question with a few sub questions:
What is the difference between unpacked vs packed array?
Are packed arrays more space efficent, how much so?
Are packed arrays more time ...
26
votes
4answers
1k views
Is it possible to speed up ContourPlot on multi-core machines?
It is not very difficult to face a function for which ContourPlot works too slow. And it seems natural that this function can be parallelized well. Anyway, naive ...
25
votes
3answers
907 views
xkcdConvert routines perform slower in Mathematica 9
I am experiencing some rather large performance decreases in Mathematica version 9.0 using the xkcd-styled plotting routines.
I had used the xkcdConvert code ...
23
votes
2answers
1k views
How can I improve the speed of eigenvalue decompositions for large matrices?
I often need to compute the eigenvalues of large matrices, and I invariably resort to MATLAB for these, simply because it is much faster. I'd like to change that, so that I can work entirely inside my ...
22
votes
6answers
773 views
Faster alternatives for DayOfWeek
It has been noticed on several occasions that DayOfWeek function is rather slow when applied to a large list of dates, e.g. in this recent question. What faster ...
21
votes
1answer
340 views
How are MemberQ and FreeQ so fast?
I would like to understand the implementation that allows MemberQ and FreeQ to be as fast as they are.
I noticed this thanks to ...
21
votes
1answer
2k views
Finding long strings of identical digits in transcendental numbers
Introduction
Describing the three main streams of present-day mathematical philosophy (formalism, Platonism and intuitionism) in a well-known book, The Emperor's New Mind, R. Penrose says:
...it ...
20
votes
2answers
272 views
Are there guidelines for avoiding the unpacking of a packed array?
Packed arrays are very useful because they save memory and generally allow speedier and more efficient calculation times. If a list of data currently stored as a packed array is unpacked, it can slow ...
20
votes
2answers
301 views
How can I ensure that I am constructing patterns in the most efficient way possible?
Inspired by this answer, I am interested to know if there are best practices or rules of thumb for constructing patterns, for example for use in function definitions (...
20
votes
3answers
840 views
How to read data file quickly?
I have a tab separated value file with 10 million rows each of which has three tab separated values. The first value is a string, the second an integer, and the third another string. How to read ...
20
votes
2answers
584 views
Can a Trie be implemented efficiently?
What is the most space and time efficient way to implement a Trie in Mathematica?
Will it be practically faster than what is natively available in appropriate cases?
19
votes
6answers
952 views
How to improve the performance of solutions to Project Euler (#39)?
This is the problem 39 of Project Euler, which I asked in the chat room two days ago. My original code runs as slowly as snails, and finally I got two answers from JM and Rojo. Unfortunately, both of ...
19
votes
5answers
802 views
Making fractals with Mathematica
I recently saw this post on math.stackexchange and was curious as to how to generate the image in Mathematica. I tried the following naive approach; however, it is extremely slow.
...
19
votes
3answers
386 views
Is there a faster way to create a matrix of indices from ragged data?
I have data that is given as a list of ordered pairs mixed with scalars. The pairs can contain infinite bounds. My goal is to convert the data into an index used in future computations.
...
18
votes
6answers
773 views
efficient way to count the number of zeros at the (right) end of a very large number
If I want to count the number of zeros at the (right) end of a large number, like $12345!$, I can use something like:
Length[Last[Split[IntegerDigits[12345!]]]]
...
18
votes
4answers
497 views
Efficient circular buffer?
I wish to create an efficient circular buffer. That is, I wish to keep a fixed length list while appending a new value and dropping the oldest, repeatedly.
As we know lists in Mathematica are ...
18
votes
1answer
372 views
Transferring a large amount of data in parallel calculations
This question is inspired by one of @whuber's answers
Consider the following code:
...
17
votes
3answers
346 views
How to efficiently find positions of duplicates?
Is there an efficient way to find the positions of the duplicates in a list?
I would like the positions grouped according to duplicated elements. For instance, given
...
17
votes
1answer
252 views
Is there an NDSolve`ProcessEquations analog for NIntegrate?
NDSolve has an interface for repeatedly solving an equation with different initial conditions without having to analyze the equation and set up the solving algorithm each time. This can improve ...
17
votes
1answer
343 views
How can I effectively performance-profile Dynamics?
How to optimize performance-wise a complex dynamic expression (like a GUI) with lots of dynamically updated components, which require different update-frequencies?
I was hesitant to ask this question ...
17
votes
2answers
314 views
Parallelize evaluation of function with memoization
I have a complicated function that I need multiple times, so I want to memoize it and have the first evaluation done in parallel. Unlike in my example below it's not a continuous function, so ...
16
votes
6answers
524 views
Split a string at specific positions
Given a string of alphanumerical characters, how to split it simply and quickly at the center of continuous letter-substrings? Is there an elegant and fast solutions out there in the "computational ...
16
votes
3answers
612 views
Reading periodic elements from a large file
I have a large binary data file (big endian) with 100+ million "rows" of 11 elements, combination of floats and integers.
This is the format:
...
16
votes
3answers
845 views
Data fitting with Image processing feature detection
I have some 2D data that once plotted looks like the following
...
16
votes
4answers
384 views
Getting lengths of sublists that sum to more than one
I have a long list of numbers, and I'm interested in finding the length of each sublist that totals to more than 1, so for the list
...
15
votes
4answers
270 views
Function that caches when it returns unevaluated
How would you go about implementing a function that caches not the result but the fact that it didn't match a pattern? So that it doesn't waste time recomputing a complex time-consuming pattern ...
15
votes
2answers
493 views
Efficient way of finding shortest distance between two sets of points in mathematica
I have two sets of 3D points, say
a = RandomReal[1, {10, 3}];
b = RandomReal[1, {10, 3}];
I wanna find the first N pairs that have shortest distance between the ...
15
votes
1answer
201 views
Adding three integer sparse matrices is very slow. Adding only two is fast
Adding more than two sparse matrices in one step in Mathematica 9 is very slow (in fact I couldn't even wait for it to finish).
Here's an example. Let's generate a large sparse matrix:
...
15
votes
0answers
352 views
Fast Spherical Harmonics radiative transfer
This is a rather specific question and I apologize for spamming you with some lengthy code. But it could be interesting for some reader and maybe you can help out, so please bear with me.
I am using ...
14
votes
4answers
877 views
Efficiently generating n-D Gaussian random fields
I am interested in an efficient code to generate an $n$-D Gaussian random field (sometimes called processes in other fields of research), which has applications in cosmology. I wrote the following ...
14
votes
4answers
270 views
How to remove duplicates from set of machine precision 2D points?
I have a set of 2D points with machine precision coordinates. I need to remove all duplicates. Performance is important.
This is the most obvious fast solution:
...
14
votes
2answers
489 views
Fast Spherical Linear Interpolation of list of quaternions
An accurate way to interpolate between two quaternions is to use Spherical Linear Interpolation (Slerp) because it preserves the unit length, whereas straightforward linear interpolation does not, as ...
14
votes
2answers
474 views
Using the Krylov method for Solve: Speeding up a SparseArray calculation
I'm trying to implement this Total Variation Regularized Numerical Differentiation (TVDiff) code in MMA (which I found through this SO answer): essentially I want to differentiate noisy data. The full ...
13
votes
3answers
358 views
Select performance
I am fairly new to mathematica and working my way thru Paul Wellins book on Mathematica programming so experimenting with various language constructs. I was experimenting with ...
13
votes
3answers
270 views
Quick multiple selections from a list
What is the fastest way to make multiple selections from a list? Compiled methods included.
For example, here are two methods for selecting a subset, compared:-
...
13
votes
3answers
309 views
Considerations when determining efficiency of Mathematica code
I have two segments of code that do the same thing and I want to determine the which is more efficient.
What are the considerations when determining efficiency of Mathematica code?
...
13
votes
2answers
304 views
Fastest way to measure Hamming distance of integers
I am looking for a fast and robust way to calculate the Hamming distance of integers. The Hamming distance of two integers is the number of matching bits in their binary representations. I expect that ...
13
votes
4answers
484 views
How to collect result continuously (interruptible calculation) when running parallel calculations?
This is the most common pattern to compute a table of results:
Table[function[p], {p, parameters}]
(regardless of how it's implemented, it could be a ...
