Mathematica Debuggability

Question

One of the things that I really dislike about Mathematica is its lack of debuggability compared to many other programming languages. Some of the problems arises from the functional nature of Mathematica. However, some of them arises from the current state of Mathematica's development environment and might be improved in later versions. Here is an incomplete list of reasons why Mathematica is less-debuggable compared to other languages. I guess some of you may disagree with some of the reasons below, but most of you agree that such a problem exists.

1. In Mathematica, function chaining is a common practice (by function chaining I mean calls like f[g[h[x]]]). That is, fewer variables are introduced to the programs. As a result, there are fewer intermediate points to check for the expected values. It makes it difficult to narrow down the scope while searching for a problem.
2. In Mathematica iterative constructs such as For and While are usually replaced by block constructs such as Table and Array. Therefore, while in imperative languages one can narrow down the focus to a single iteration of a loop (and even narrower), in Mathematica, it's not an easy job.

3. Mathematica doesn't Fail-fast. For instance, if you have a call on an undefined function f, it will continue computing. Note that, it may be the desired behaviour in some cases but generally it reduces debuggability. The problem becomes more severe when you pass such a function to another function or a function chain. Another (maybe more complicated) instanc follows:

   Test[i_Integer]=2 i;

   The function Test is supposed to be called on Integers only, but it doesn't fail if it is called on other types. In some cases we can prevent such an issue by replacing the above line with:

   Test[_] := Throw["Incorrect function call"];

   Test[i_Integer]=2 i;

4. Anonymous functions and lazy evaluation are commonly used in Mathematica.

5. Unlike many other languages such as Java, Mathematica doesn't produce a full stack trace while encounters an exception. It makes it difficult to trace the problem when an exception is thrown (there might be an option to turn on the stack traces that I am unaware of).

6. Mathematica keeps the state of notebook between different runs. One can define a function or variable and evaluate the cell. Then delete the cell and the function or variable is still defined and can be used.

7. Variables and functions are defined in the global context by default unless otherwise stated (e.g., using Module or With functions), while in many other languages variables are defined locally by default. Different notebooks share the same context by default that adds to the complexity of debugging. This is against the principle of least privilege.

Now, the question is: what are the good practices to prevent errors in Mathematica and what are the best debugging techniques if you encountered a problem. I think that's a good idea to have one solution per answer so we can discuss about it easily.

Answer 1

While I wait for better answers from some very knowledgeable people in the matter on the site, I'll write what I'm thinking...

I think that most of your problems are due to lack of practice with functional thinking rather than lack of debugability itself.

1. I think one that on the contrary, one of the advantages of programming functionally is that the state of the program is kept on the stack. If you know where you are, you know everything. In imperative programming its harder to know all the variables that keep the state at any point to understand the behaviour. If goto and label are worse for following the flow of a program versus normal loop structures, then functional seems another step ahead. You can always put a function like @LeonidShiffrin's ShowIt where you want to see what's going on, or something similar to print the stack (see the function Stack[_]):

   SetAttributes[ShowIt, HoldAll];
   ShowIt[code_] := 
      Module[{y}, 
         Print[ToString[Unevaluated[code]], " = ", y = code]; 
         y]
   

2. You answered yourself. Always remember that in Mathematica, code and expressions are the same, so you can always make a "function" that saves you all the boilerplate code if you think that certain solution can be done but its hard and makes you type a lot. (You could even take it to the extreme of brutally overload SetDelayed to always add a func[___]:=Throw, but I don't think that's recomended, hehe)

As to how to make the debugging simpler, well...

• Build up from short declarative functions. Comment a lot, Test them before going on chaining 11 of them.

• Use Messages

• Check out the debugger, at least the Workbench one. They let you add breakpoints, see the stack, and even break on messages

• Use functions such as that ShowIt from Leonid, or make your own.

• Learn to use Trace and family. I really like a version of it that I think I took from a post by @WReach

  TraceViewShort[expr_] := 
   Module[{steps = {}, stack = {}, pre, post}, 
    pre[e_] := (stack = {steps, stack}; steps = {}); 
    post[e_, r_] := (steps = First@stack~Join~{{e, steps, HoldForm[r]}};
       stack = stack[[2]]); SetAttributes[post, HoldAllComplete]; 
    TraceScan[pre, expr, ___, post]; 
    DynamicModule[{focus, show, substep, enter, exit}, focus = steps; 
     substep[{e_, {}, _}, _] := {Null, e, 
       Style["inert", {Italic, Small}]}; 
     substep[{e_, _, r_}, 
       p_] := {Button[Style["show", Small], enter[p]], e, 
       Style[Row[{"-> ", r}], Small]}; 
     enter[{p_}] := PrependTo[focus, focus[[1, 2, p]]]; 
     exit[] := focus = Drop[focus, 1]; 
     show[{e_, s_, r_}] := 
      Column[{Grid[{{"Expression", 
           Column@Reverse@focus[[All, 1]]}, {Column[{"Steps", 
             focus /. {{_} :> Sequence[], _ :> 
                Button["Back", exit[], ImageSize -> Automatic]}}], 
           Grid[MapIndexed[substep, s], Alignment -> Left]}, {"Result", 
           Column@focus[[All, 3]]}}, Alignment -> Left, Frame -> All, 
         Background -> {{LightCyan}}]}]; Dynamic@show@focus[[1]]]]
  SetAttributes[TraceViewShort, {HoldAllComplete}]
  

• Check the function arguments. In some cases it may make more sense to use your f[___]:=Throw and in others leave it unevaluated

• Consider Assert, it works well with the debugger and doesn't make your code slower because you can turn it off...

Anyway, Mathematica allows you to do almost anything you want. Most of the things you said are not Mathematica limitations but, in the worst case, limitations of the way you're using it. Sometimes that extra freedom has the disadvantage of not giving you guidelines as to what's best.

Answer 2

While I agree that the debugging tools could have been better developed by now, let me just throw in a few notes and links.

• Function chaining (f[g[h[...]]]): I'd argue that this is a good thing. Why:

  • Functions return expressions, which are immutable. You don't introduce as much state (or at all), as in imperative languages. This makes it easier to debug any given function, because functions do not depend on the environment.
  • Functions can be much easier developed and tested, since they take and return expressions.
  • This makes it possible to have better means for composition than when functions produce side effects.

• It is just as easy to narrow down the focus in Table or Array etc - just make them smaller for your tests. In fact, I'd argue that loops are way more error-prone, and it is no coincidence that Java introduced for-each iteration, Iterator objects etc - all of those essentially to automate the loops and exclude certain common classes of errors.

• Regarding Fail-fast: I recommend reading my post where I desribe some common error-handling practices used in Mathematica code - this should address some of your concerns. To put it short, you can always return $Failed or throw an exception, or use Assertions. I also recommend this discussion.

• Lazy evaluation and labda-functions should not present any problems, as long as they are not mixed with too stateful code (mixing state and behavior is always a bad idea I think).

• As for the stack traces, the debug function I developed here, does return something similar to stack traces, and, at least in my experience, is very helpful. It is triggered by the first message issued during the program's execution.

There were many debugging-related discussions on Mathematica-related forums. Here is one link off the top of my head: How to debug when writting small or big codes using Mathematica? workbench? mma debugger? or something else?

On a personal note, I seldom have serious debugging problems with my code. I think, Mathematica makes it possible to write easily testable and modular code which requires very little debugging. So, my advice to you would be the following: try to write your code clean, modular and using all the powerful functional, rule-based, and meta-programming abstractions Mathematica offers - the best debugging is to write code that needs no debugging :)