More structure in Source Code/Notebooks

Question

Mathematica is great for small and quick projects and has a great syntax. However as soon as a project grows I run into trouble. How do you scale and maintain projects in Mathematica? What is beyond the interactive notebook style?

So far I

1. get something going in different cells
2. copy and paste it into one cell
3. define a function around it

If I want change something I usually mess something up and then have to break up the code again to find the error. In C++ you obtain the structure by thinking object oriented...

Answer 1

I typically go from prototyping some code to a structured functional interface through pretty much the steps you described. Starting out with

 a=3;
 b=a^2+3;
 Sin[a]+Cos[b];

Or similar string of random things across a couple of cells, then I join up the elements into one cell, wrap them in Module clear the kernel and verify that it runs as is:

 Module[{a,b},
   a=3;
   b=a^2+3;
   Sin[a]+Cos[b]
 ]

This tests if you have everything you where relying on captured in the module. I then move some arguments out to a function interface.

 f::usage="f[a_] does something!";
 f[a_]:= Module[{b},
   b=a^2+3;
   Sin[a]+Cos[b]
 ]

If I have multiple functions that interact in ways that are more complicated and need more documentation, I will sometimes use custom data structures like CoinList[1,2,3] and define my functions as UpValues of this. This way it's easier to distinguish if for instance you start getting nested structures of lists and your particular data, and rather then just defining functions to take lists as arguments you get them specialized on your data.

howMuchMoney[a_CoinList] ^:= Total[List@@a]

And also when you find some old code and wonder "What can I do with CoinLists?" you can just type out ??CoinList and all the definitions show up.

If it gets more complicated then that, I'll fire up workbench and write docpages for functions and structures.

Answer 2

My 2 pesos:

Build functions from functions from smaller functions etc etc. Don't let your functions grow big. If you needed to break down a function too much in order to change one little thing, then probably that little thing could have been inside a smaller function, included in the big one.

You can use sample data while you build them and test them, but after getting it right, just keep the function. Small functions are easy to test, you can reuse them, are clearer to read and get loaded instantaneously because no computation is actually done.

When you have a bunch of them with some commonality, put them in a separate notebook, or at least a cell group (possibly an initialization cell group). When you don't feel you'll be touching them much again, if it's worth it, take a minute to build a package.

Use cell groups, sections, subsections, to collapse code and navigate it more or less easily, if your notebook gets big.

Consider Workbench for bigger projects

Answer 3

The approach you've outlined in your question is pretty much the one I use. When I do a large project, I make a master notebook which focuses on presenting the finished code needed to carry out whatever computation I am trying to perform, but doesn't show any of the development and testing that went into the components. I also have one or more (usually more) auxiliary notebooks where I carry out the development of the functions and data structures that go into the main project notebook. All the notebooks relating to one project live together in a project folder.

Here is an an example of an auxiliary notebook I worked on today.

---

Test data

data = Table[{i, j}, {i, 5}, {j, 10, 50, 10}] /. {i_, j_} -> {i, j,
1./(i + 0.1 j)}

{{{1, 10, 0.5}, {1, 20, 0.333333}, {1, 30, 0.25}, {1, 40, 0.2}, {1, 50, 0.166667}},
{{2, 10, 0.333333}, {2, 20, 0.25}, {2, 30, 0.2}, {2, 40, 0.166667}, {2, 50, 0.142857}},
{{3, 10, 0.25}, {3, 20, 0.2}, {3, 30, 0.166667}, {3, 40, 0.142857}, {3, 50, 0.125}},
{{4, 10, 0.2}, {4, 20, 0.166667}, {4, 30, 0.142857}, {4, 40, 0.125}, {4, 50, 0.111111}},
{{5, 10, 0.166667}, {5, 20, 0.142857}, {5, 30, 0.125}, {5, 40, 0.111111}, {5, 50, 0.1}}}

A label for identifying the row/column parameters

topLeftLabel = Module[{slash, rho, x, zero},
   slash = Line[{{0., 2.}, {2., 0.}}];
   rho = Text["\[CurlyRho]", {0.5, 0.5}];
   x = Text[Style["x", Italic], {1.5, 1.5}];
   zero = Text[Style[0, 7], {1.5, 1.5}, {-2., 0.9}];
   Graphics[{slash, rho, x, zero}, ImageSize -> 18]]

Preparing the data for tabulation

rtData = data[[All, 1, 1]]

{1, 2, 3, 4, 5}

topData = data[[1, All, 2]]

{10, 20, 30, 40, 50}

values = data[[All, All, 3]]

{{0.5, 0.333333, 0.25, 0.2, 0.166667},
{0.333333, 0.25, 0.2, 0.166667, 0.142857},
{0.25, 0.2, 0.166667, 0.142857, 0.125},
{0.2, 0.166667, 0.142857, 0.125, 0.111111},
{0.166667, 0.142857, 0.125, 0.111111, 0.1}}

topLbl = Prepend[Item[#, Alignment -> {Center, Bottom}] & /@ topData, topLeftLabel]

{label-graphic, 10, 20, 30, 40, 50}

rows = Prepend[MapThread[Prepend, {values, rtData}], topLbl]

{{label-graphic, 10, 20, 30, 40, 50},
{1, 0.5, 0.333333, 0.25, 0.2, 0.166667},
{2, 0.333333, 0.25, 0.2, 0.166667, 0.142857},
{3, 0.25, 0.2, 0.166667, 0.142857, 0.125},
{4, 0.2, 0.166667, 0.142857, 0.125,0.111111},
{5, 0.166667, 0.142857, 0.125, 0.111111, 0.1}}

Making the tabulation

Grid[rows, Alignment -> ".", Dividers -> {All, {True, True, {False}, True}}]

Tabulate, a function to tabulate data 3-tuples

With[{lblSize = 18},
   With[{default = Graphics[{White, Rectangle[]}, ImageSize -> lblSize]},
      tabulate[data_List, topLeftLabel_Graphics: default] :=
         Module[{rtData, topData, values, topLbl, rows},
            rtData = data[[All, 1, 1]];
            topData = data[[1, All, 2]];
            values = data[[All, All, 3]];
            topLbl =
               Prepend[Item[#, Alignment -> {Center, Bottom}] & /@ topData,
                  topLeftLabel];
            rows =
               Prepend[MapThread[Prepend, {values, rtData}], topLbl];
            Grid[rows, Alignment -> ".",
               Dividers -> {All, {True, True, {False}, True}}]]]]

Unit Tests

tabulate[data]

tabulate[data, topLeftLabel]

---

The code creating the graphic topLeftLabel and the function tabulate will be pasted into the main project notebook.

A further pair of organizational tools that I use are cell tags and hyperlinks. These make it easy to jump between related parts of a notebook that are physically distant. I will often place a table-of-contents made up of hyperlinks to Section cell tags at the beginning of my main notebook.

My projects so far have never been more than modestly large. I have never worked on one so large that I felt the need for Wolfram Workbench.