# Good ways to organize and document collections of mathematica notebooks?

I've now been using Mathematica for about a year, and am starting to get a somewhat unwieldy collection of notebooks. Sometimes I've figured out how to do a particular task in one of them, and want to use that again, but have to search through what I've done.

To tackle this, I've cobbled together a script that generates an html page for all the notebooks that I've checked into my source code repository (I happen to be using github to store my notebooks, latex, and other source code), and can put comments in that generated html file about what I did in each notebook, including any special techniques that I learned to accomplish the task. By way of example, I can list these with something like on this page.

From a local copy of my source repository I generate something similar, allowing a click on the various notebooks to view them with the CDF player.

I'm able to navigate my collection of notebooks well enough this way, but was curious how other people tackle the same organizational problem?

-
By a crazy directory structure and file-naming scheme, often helped by utilizing the "Search inside a file" feature on Windows. In other words, my pile of Mathematica notebooks looks much like the piles of notes on my desk... –  Eli Lansey Feb 2 '12 at 21:20
I use folders and filenames. –  David Feb 2 '12 at 21:21
+1 for citing Landau-Lifshitz –  acl Feb 2 '12 at 21:21
@verbeia, Perhaps, if Mathematica provides infrastructure for organizing and searching notebooks. –  Peeter Joot Feb 3 '12 at 0:17
I've been thinking of a system for exactly this. I was thinking of storing a database of notebook information (file path, title, tags) in a .m file and then making a notebook with a GUI for searching the database by tag and notebook contents. –  Andrew MacFie Feb 3 '12 at 3:44
show 5 more comments

There are two answers to this question, and only one has anything to do with directory structure, and that only mildly.

The first answer is to structure the notebook itself using the style groupings provided. This will greatly enhance your ability to find something within any given notebook. For instance, as a materials physicist I use a model known as tight binding, and as part of my learning about how to apply it to a particular crystal structure, I have a notebook that I treat as a lab notebook. It has three main sections: Setup, Preliminary Testing, and Full Testing all at the highest level of the hierarchy. The least organized section is the Preliminary Testing, but even there I make ample use of Text cells to record my thoughts and observations. The other two sections make full use of the hierarchy to organize and group my examinations of the tight-binding method.

The second answer is packages. If you have some code that you find yourself referring to often, put in a package. Here I tend to use a rule of three, if I have to refer to or rewrite something 3 or more times, it needs to be in a package. It is here that a directory structure is helpful. In my case, my packages are currently organized into three directories: Quantum, Utilities, and Wien2k (a software package I've written interface scripts for) for a total of 16 packages.

I try to make each package somewhat complete in that they provide a full range of services. For example, the Wien2k package for loading their crystal structure files provides a full range of functions for accessing the structured information in those files, but sadly not saving them, as I haven't had the need to write it. But, some packages are just buckets of loosely related functions, and they're perfect that way.

-
So after you have a bunch of packages, organized into directories, you begin to have essentiall the same as at the start?? –  murray Sep 1 '12 at 15:06
@murray -- good question; not too big, and I'm careful to keep Graphics/Image objects out of the Notebooks most of the time and I turn off Cell edit tracking to minimize size. My largest Notebooks of this type are only around 3MB. Indeed adding another level of hyperlinks, from one of these Notebooks to others, rather than only from the Palette to the Notebooks, would be a good idea for any large projects. I don't have so many large projects that I cannot keep individual Notebooks for them and find them easily. –  Mr.Wizard Sep 1 '12 at 19:55