Some people advise against the use of For loops in Mathematica. Why? Should I heed this advice? What is wrong with For? What should I use instead?

  • 2
    This comes up so often, I wanted to have something to link to. As always on StackExchange, other answers are welcome. – Szabolcs Jan 2 '17 at 14:40
  • 2
    As, still, someone not that fluent in Mathematica, and a longtime programmer (mostly C), this was the most useful question/answer I've read - and I have read a bunch of these! Thanks to both the questioner and the responder. – Jack Adams Jan 3 '17 at 21:45
  • 1
    Related (my own self Q&A): Alternatives to procedural loops and iterating over lists in Mathematica: "Explicit loops are often counterproductive in Mathematica, not only taking more keystrokes, but also more execution time. They are also, in my opinion, more prone to mistakes." – Mr.Wizard Jan 5 '17 at 9:30

If you are new to Mathematica, and were directed to this post, first see if you can use Table to solve your problem.


I have often told people, especially beginners, to avoid using For in favour of Do. The following is my personal opinion on why using For is harmful when learning Mathematica. If you are a seasoned Mathematica user, you won't find much to learn here. My biggest argument against For is that it hinders learning by encouraging error-prone, hard to read, and slow code.

For mimics the syntax of the for loop of C-like languages. Many beginners coming from such languages will look for a "for loop" when they start using Mathematica. Unfortunately, For gives them lots of ways to shoot themselves in the foot, while providing virtually no benefits over alternatives such as Do. Settling on For also tends to delay beginners in discovering more Mathematica-like programming paradigms, such as list-based and functional programming (Table, Map, etc.)

I want to make it clear at the beginning that the following arguments are not about functional vs procedural programming. Functional programming is usually the better choice in Mathematica, but procedural programming is also clearly needed in many situations. I will simply argue that when we do need a procedural loop, For is nearly always the worst choice. Use Do or While instead.

Use Do instead of For

The typical use case of For is iterating over an integer range. Do will do the same thing better.

  • Do is more concise, thus both more readable and easier to write without mistakes. Compare the following:

    For[i=1, i <= n, i++, 
      f[i]; g[i]; h[i]
    ]
    
    Do[ f[i]; g[i]; h[i], {i, n} ]
    

    In For we need to use both commas (,) and semicolons (;) in a way that is almost, but not quite, the opposite of how they are used in C-like languages. This alone is a big source of beginner confusion and mistakes (possibly due to muscle memory). , and ; are visually similar so it is hard to spot the mistake.

  • For does not localize the iterator i. A safe For needs explicit localization:

    Module[{i},
      For[i=1, i <= n, i++, ...]
    ]
    

    A common mistake is to overwrite the value of a global i, possibly defined in an earlier input cell. At other times i is used as a symbolic variable elsewhere, and For will inconveniently assign a value to it.

    In Do, i is a local variable, so we do not need to worry about these things.

  • C-like languages typically use 0-based indexing. Mathematica uses 1-based indexing. for-loops are typically written to loop through 0..n-1 instead of 1..n, which is usually the more convenient range in Mathematica. Notice the differences between

    For[i=0, i < n, i++, ...]
    

    and

    For[i=1, i <= n, i++, ...]
    

    We must pay attention not only to the starting value of i, but also < vs <= in the second argument of For. Getting this wrong is a common mistake, and again it is hard to spot visually.

  • In C-like languages the for loop is often used to loop through the elements of an array. The literal translation to Mathematica looks like

    For[i=1, i <= n, i++,
      doSomething[array[[i]]]
    ]
    

    Do makes this much simpler and clearer:

    Do[doSomething[elem], {elem, array}]
    
  • Do makes it easy to use multiple iterators:

    Do[..., {i, n}, {j, m}]
    

    The same requires a nested For loop which doubles the readability problems.

Transitioning to more Mathematica-like paradigms

A common beginner-written program that we see here on StackExchange collects values in a loop like this:

list = {};
For[i=1, i <= n, ++i,
  list = Append[list, i^2]
]

This is of course not only complicated, but also slow ($O(n^2)$ complexity instead of $O(n)$). The better way is to use Table:

Table[i^2, {i, n}]

Table and Do have analogous syntaxes and their documentation pages reference each other. Starting out with Do makes the transition to Table natural. Moving from Table to Map and other typical functional or vectorized (Range[n]^2) constructs is then only a small step. Settling on For as "the standard looping construct" leaves beginners stuck with bad habits.

Another very common question on StackExchange is how to parallelize a For loop. There is no parallel for in Mathematica, but there is a ParallelDo and more importantly a ParallelTable. The answer is almost always: design the computation so that separate steps of the iteration do not access the same variable. In other words: just use Table.

More general versions of For

For is of course in some ways more flexible than Do. It can express a broader range of iteration schemes. If you need something like this, I suggest just using While instead.

When we see for, we usually expect either a simple iteration through an integer range or through an array. Doing something else, such as modifying the value of the iterator in the loop body is unexpected, therefore confusing. Using While signals that anything can happen in the loop body, so the readers of the code will watch out for such things.

When is For appropriate?

There are some cases when For is useful. The main example is translating code from other languages. It is convenient to be able to translate analogous for loops, and not have to think about what may be broken by immediately translating to a Do or a Table (e.g. does the loop modify the iterator in the body?). Once the translated code works fine, it can be rewritten gradually.

There are existing questions on this, which also discuss other cases:

Summary

The problem with For is that it hinders learning and makes it very easy for beginners to introduce mistakes into their code.

If you are new to Mathematica, my advice is to forget that For exists, at least for a while. You can always accomplish the very same things with Do and While—use them instead. Very often you will be able to replace Do with a Table or even a vectorized expressions. This will help you learn to write effective Mathematica code faster.

If you are unsure about a use of For, then ask yourself: do I see a reason why For is clearly better here than Do or While? If not, don't use it. If yes, you may have found one of the rare good use cases.

  • 5
    @Feyre It is, in several ways. And the jump from Table to vectorized computation is quite straightforward—another reason to use Do, which then leads to Table, which then leads to the paradigm you show. But there is only so much space in one post. I wrote this up so I can link to it when people ask why they are told to avoid For. – Szabolcs Jan 2 '17 at 15:05
  • 2
    +1, but one nit. I've never understood how Table is considered more inline with a "functional" paradigm than For. There are obvious utilities with using Table/Do over For, but other languages have used similar forms for years, usually expressed as ForEach and c++ has gotten in on the act via the ranged for. – rcollyer Jan 2 '17 at 16:37
  • 2
    Avoid For, Do is better. Avoid Do, Table is better. Avoid Table, Range^2 is better. When is it good enough? – DepressedDaniel Jan 2 '17 at 18:45
  • 2
    @DepressedDaniel when our computations become fast enough to solve the most basic problems in Ramsey Theory ;) let's start with $R(5,5)$ and work from there... – Brevan Ellefsen Jan 2 '17 at 19:05
  • 4
    "For does not localize the iterator i" -- that alone for me is a sufficient argument to avoid For. All the rest are good points, but mostly about style / readability whereas this one actually breaks the functional pattern (I strive to have as few assignments as possible in general) and may lead to hard-to-find bugs (as opposed to simple typo's that Mathematica will tell you about). – CompuChip Jan 3 '17 at 13:17

Illustration of the timings required to compute the squares i^2 from i=1 to i=10^n for n=1, 2, ..., 7 with the use of For, While, Do, Table, and Range.

for = Table[
  Module[{i},
   For[i = 1, i <= 10^n, i++, i^2] // AbsoluteTiming // First
   ]
  , {n, 1, 7}]

while = Table[
  Module[{i},
   i = 1; While[i <= 10^n, i^2; i++] // AbsoluteTiming // First
   ]
  , {n, 1, 7}]

do = Table[Do[i^2, {i, 10^n}] // AbsoluteTiming // First, {n, 1, 7}]

table = Table[
  Table[i^2, {i, 10^n}]; // AbsoluteTiming // First, {n, 1, 7}]

range = Table[Range[10^n]^2; // AbsoluteTiming // First, {n, 1, 7}]

(By the way, look how concise are the codes of Do, Table and Range compared to For and While.)

The timings for 10^7 squares (i.e., n=7):

Last /@ {for, while, do, table, range}

{7.32907, 8.23668, 2.44558, 0.132735, 0.036395}

And a plot (vertical axis in log-scale):

ListLogPlot[{for, while, do, table, range}, Frame -> True, 
 PlotRange -> All, Joined -> True, ImageSize -> 400, 
 FrameLabel -> {"n", "Log[AbsoluteTiming] (sec)"}, 
 PlotLabels -> {"For", "While", "Do", "Table", "Range"}]

enter image description here

Do is about $3\times$ faster than For/While; for this particular application, one could (and should) employ Table/Range, which are two orders of magnitude faster than For.

  • 1
    Not sure that's a fair comparison, because: (1) Do, for example, throws away everything calculated (returning Null) whereas Table, for example, actually returns the list of squares; and (2) calculating i^2 is relatively trivial and misleadingly magnifies the effect of the looping construct used. – murray Jan 2 '17 at 16:51
  • @murray I've made it a community wiki; feel free to improve. – corey979 Jan 2 '17 at 16:53
  • @murray In general, you are right: one of the things the benchmark shows is the performance of the looping construct itself, as i^2 is so fast. In more typical uses it is the speed of looping that is negligible compared to the loop body. But there's more here: Table is fast because of auto-compilation. Auto-compilation is possible because of the functional nature of Table: each element of the table is computed independently of the others and usually without side effects. – Szabolcs Jan 3 '17 at 11:32
  • @murray In contrast, each evaluation done by Do will have side effects, otherwise it would be pointless. Usually, that side effect is changing the same global variable (e.g. accumulating a sum), which makes the evaluations non-independent. Thus Do does not auto-compile. Similarly, Table is immediately parallelizable while Do isn't. My original answer wasn't simply about performance. There's much more to it: readability, manageability, etc. But I think it is true that using For can significantly hinder performance. Of course this applies to Mathematica only, not other languages. – Szabolcs Jan 3 '17 at 11:32
  • 1
    +1 for actual science, even if a microbenchmark-y one – vaxquis Jan 3 '17 at 14:40

The functional paradigm, exemplified by this code:

Map[(#^2) &, Range[10^7]] // AbsoluteTiming

will usually result in the fastest execution because it takes advantage of the architecture of the machine. Both the CPU and the memory are optimized for sequential access, so when you pass a function over a list of data to transform that data, the code stays in one place, taking advantage of locality (no code-cache misses), and the data is accessed as one continuous stream of bytes. The above line of code takes 0.281 seconds to complete on my computer, while the line below ran for well over an hour and only produced a list 1,190,218 elements long:

out = {};
For[i = 1, i <= 10^7, ++i, AppendTo[out, i^2]] // AbsoluteTiming
  • 3
    The fastest way is in fact using array arithmetic (i.e. "vectorization"): Range[10^7]^2. Map needs to evaluate Mathematica code for each element, which is slower than handling the entire array with a low-level implementation. To be precise, Map will auto-compile #^2&, which gives a considerable speedup. But it is still not competitive with parallelized array arithmetic that makes use of SIMD instructions. – Szabolcs Apr 5 '17 at 16:59
  • 1
    But generally you are correct. If this were not something that can be expressed with basic vector arithmetic, then Map would be a good way. It is only purely numerical operations on packed arrays that will benefit for vectorization. – Szabolcs Apr 5 '17 at 17:31

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.