Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Mathematica is great for small and quick projects and has a great syntax. However as soon as a project grows I run into trouble. How do you scale and maintain projects in Mathematica? What is beyond the interactive notebook style?

So far I

  1. get something going in different cells
  2. copy and paste it into one cell
  3. define a function around it

If I want change something I usually mess something up and then have to break up the code again to find the error. In C++ you obtain the structure by thinking object oriented...

share|improve this question
    
One suggestion is to use the Workbench for large projects. –  b.gatessucks Dec 20 '12 at 21:30
    
This is a bit open ended so just some odds and ends for consideration: workbench as mentioned by @b.gatessucks, create packages, use "check balance" when you mess something up after changes and try to figure out where it occurred. I'm sure others will add many more. –  Mike Honeychurch Dec 20 '12 at 21:35
    
@warsaga Could you paste a small example of how you go from unctructured to structured code and where you think it gets confusing? I think you are looking for something resembling OOP for all the wrong reasons. –  jVincent Dec 21 '12 at 1:03
    
Notebooks are not the right software development tool for larger projects, at least not without some additional framework on top of the standard notebook interface. Generally, you want source code (packages), and some IDE (Workbench is a good choice) to manage them. Notebooks contain too much meta-data. –  Leonid Shifrin Dec 21 '12 at 10:18
    
@Leonid Personally, the main problem I find in trying to develop purely in notebooks is that there is no way to navigate your code (unless you write one, which admittedly isn't that hard). metadata and the rest is a problem for version control, but I find the lack of basic navigational aids makes me switch to the workbench long before I get anywhere near the size for which I'd use any versioning system. And while the workbench is ok, it would have been nice if it had autocompletion of templates (for example). As it is, it's merely less bad than programming in the frontend. –  acl Dec 21 '12 at 12:24

3 Answers 3

My 2 pesos:

Build functions from functions from smaller functions etc etc. Don't let your functions grow big. If you needed to break down a function too much in order to change one little thing, then probably that little thing could have been inside a smaller function, included in the big one.

You can use sample data while you build them and test them, but after getting it right, just keep the function. Small functions are easy to test, you can reuse them, are clearer to read and get loaded instantaneously because no computation is actually done.

When you have a bunch of them with some commonality, put them in a separate notebook, or at least a cell group (possibly an initialization cell group). When you don't feel you'll be touching them much again, if it's worth it, take a minute to build a package.

Use cell groups, sections, subsections, to collapse code and navigate it more or less easily, if your notebook gets big.

Consider Workbench for bigger projects

share|improve this answer

The approach you've outlined in your question is pretty much the one I use. When I do a large project, I make a master notebook which focuses on presenting the finished code needed to carry out whatever computation I am trying to perform, but doesn't show any of the development and testing that went into the components. I also have one or more (usually more) auxiliary notebooks where I carry out the development of the functions and data structures that go into the main project notebook. All the notebooks relating to one project live together in a project folder.

Here is an an example of an auxiliary notebook I worked on today.


Test data

data = Table[{i, j}, {i, 5}, {j, 10, 50, 10}] /. {i_, j_} -> {i, j,
1./(i + 0.1 j)}

{{{1, 10, 0.5}, {1, 20, 0.333333}, {1, 30, 0.25}, {1, 40, 0.2}, {1, 50, 0.166667}},
{{2, 10, 0.333333}, {2, 20, 0.25}, {2, 30, 0.2}, {2, 40, 0.166667}, {2, 50, 0.142857}},
{{3, 10, 0.25}, {3, 20, 0.2}, {3, 30, 0.166667}, {3, 40, 0.142857}, {3, 50, 0.125}},
{{4, 10, 0.2}, {4, 20, 0.166667}, {4, 30, 0.142857}, {4, 40, 0.125}, {4, 50, 0.111111}},
{{5, 10, 0.166667}, {5, 20, 0.142857}, {5, 30, 0.125}, {5, 40, 0.111111}, {5, 50, 0.1}}}

A label for identifying the row/column parameters

topLeftLabel = Module[{slash, rho, x, zero},
   slash = Line[{{0., 2.}, {2., 0.}}];
   rho = Text["\[CurlyRho]", {0.5, 0.5}];
   x = Text[Style["x", Italic], {1.5, 1.5}];
   zero = Text[Style[0, 7], {1.5, 1.5}, {-2., 0.9}];
   Graphics[{slash, rho, x, zero}, ImageSize -> 18]]

PNG-LABEL

Preparing the data for tabulation

rtData = data[[All, 1, 1]]

{1, 2, 3, 4, 5}

topData = data[[1, All, 2]]

{10, 20, 30, 40, 50}

values = data[[All, All, 3]]

{{0.5, 0.333333, 0.25, 0.2, 0.166667},
{0.333333, 0.25, 0.2, 0.166667, 0.142857},
{0.25, 0.2, 0.166667, 0.142857, 0.125},
{0.2, 0.166667, 0.142857, 0.125, 0.111111},
{0.166667, 0.142857, 0.125, 0.111111, 0.1}}

topLbl = Prepend[Item[#, Alignment -> {Center, Bottom}] & /@ topData, topLeftLabel]

{label-graphic, 10, 20, 30, 40, 50}

rows = Prepend[MapThread[Prepend, {values, rtData}], topLbl]

{{label-graphic, 10, 20, 30, 40, 50},
{1, 0.5, 0.333333, 0.25, 0.2, 0.166667},
{2, 0.333333, 0.25, 0.2, 0.166667, 0.142857},
{3, 0.25, 0.2, 0.166667, 0.142857, 0.125},
{4, 0.2, 0.166667, 0.142857, 0.125,0.111111},
{5, 0.166667, 0.142857, 0.125, 0.111111, 0.1}}

Making the tabulation

Grid[rows, Alignment -> ".", Dividers -> {All, {True, True, {False}, True}}]

PNG-TABLE-1

Tabulate, a function to tabulate data 3-tuples

With[{lblSize = 18},
   With[{default = Graphics[{White, Rectangle[]}, ImageSize -> lblSize]},
      tabulate[data_List, topLeftLabel_Graphics: default] :=
         Module[{rtData, topData, values, topLbl, rows},
            rtData = data[[All, 1, 1]];
            topData = data[[1, All, 2]];
            values = data[[All, All, 3]];
            topLbl =
               Prepend[Item[#, Alignment -> {Center, Bottom}] & /@ topData,
                  topLeftLabel];
            rows =
               Prepend[MapThread[Prepend, {values, rtData}], topLbl];
            Grid[rows, Alignment -> ".",
               Dividers -> {All, {True, True, {False}, True}}]]]]
Unit Tests
tabulate[data]

PNG-TABLE-2

tabulate[data, topLeftLabel]

PNG-TABLE-1


The code creating the graphic topLeftLabel and the function tabulate will be pasted into the main project notebook.

A further pair of organizational tools that I use are cell tags and hyperlinks. These make it easy to jump between related parts of a notebook that are physically distant. I will often place a table-of-contents made up of hyperlinks to Section cell tags at the beginning of my main notebook.

My projects so far have never been more than modestly large. I have never worked on one so large that I felt the need for Wolfram Workbench.

share|improve this answer

I typically go from prototyping some code to a structured functional interface through pretty much the steps you described. Starting out with

 a=3;
 b=a^2+3;
 Sin[a]+Cos[b];

Or similar string of random things across a couple of cells, then I join up the elements into one cell, wrap them in Module clear the kernel and verify that it runs as is:

 Module[{a,b},
   a=3;
   b=a^2+3;
   Sin[a]+Cos[b]
 ]

This tests if you have everything you where relying on captured in the module. I then move some arguments out to a function interface.

 f::usage="f[a_] does something!";
 f[a_]:= Module[{b},
   b=a^2+3;
   Sin[a]+Cos[b]
 ]

If I have multiple functions that interact in ways that are more complicated and need more documentation, I will sometimes use custom data structures like CoinList[1,2,3] and define my functions as UpValues of this. This way it's easier to distinguish if for instance you start getting nested structures of lists and your particular data, and rather then just defining functions to take lists as arguments you get them specialized on your data.

howMuchMoney[a_CoinList] ^:= Total[List@@a]

And also when you find some old code and wonder "What can I do with CoinLists?" you can just type out ??CoinList and all the definitions show up.

If it gets more complicated then that, I'll fire up workbench and write docpages for functions and structures.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.