Project and notebook design

Question

I have been using Mathematica now for a little over a year...this started off in maybe a modern 'classic' way of having to solve symbolic mathematics and related plus the want to learn a CAS (and more) kind of program.

I bought the hands on book, and became an activeish member of this community and have even used MMA to successfully solve some of my own work problems and personal ones.

Infact MMA has become probably the one program I've ever bought that I have used every single day since purchase. Needless to say I'm hooked, but my skill level is not so far away from a novice, compared to what is possible with the software.

At my work, I'm about to be starting a fairly complicated mechanical/electrical project and I intend to do all the heavy lifting via Mathematica. The project will go something like this:

1. Initial CAD design
2. Symbolic dynamic model design
3. System Parameters verification
4. System Analysis
5. 1-4 Rinse and repeat depending on 4
6. Control Scheme designed and analysed
7. Embedded system programming for HIL prototyping and testing

Besides the CAD part of the list, all of this is intended to be done via MMA. (especially thanks for MicroControllerKit in v12)

Each one of these things in my list I have done already in what I would call messy but some what "organised" notebooks, sometimes separated from each other depending on task complexity or mashed into one giant notebook.

Typically I have been starting my notebooks roughly in this sort of format each in a different cell:

1. Needs and importing any data files
2. Assigning variable names to static things
3. Initialising any f[x_]:=
4. Initialising any params = {..}
5. Doing tasks divided by cells and title/text blocks describing what the following code does.

This becomes quickly in my opinion very crowded and hard to read as the code results and plots stretches into endless pages. On top of this sometimes errors will occur breaking huge swaths of code because of two variables having the same name or some strange thing happening requiring a kernel restart. I am unsure if my method is a proper one or not, or if things should be separated and/or imported between each other.

Being such a novice at doing "real" projects, I would like to ask:

What are the best practices for project layout, coding and organisation for large engineering projects? (or any large project)

I have read this post "Mathematica Style Guide" However this gives more of a coding style guide (which was a good read anyways). But I am looking for notebook/project planning guide/layout of best practices.

Answer 1

At a very high level, best practice would be to break the problem down into simple steps and to automate wherever possible. And document. And backup, possibly using git.

So one way to organise your problem is to confine each of the steps 1-4 to separate directories. It may be that you decide to maintain debugged functions in a common package. It may also be that you have enough functions to require many packages but in either case, MMA can accommodate your workflow.

Instead of regenerating parameters each run, reading from an MX file can save time & and confusion.

If you're generating many intermediate results, consistent nomenclature is useful, and programatically generating a series of experiments provides great control allowing easy output & retrieval of results for analysis & synthesis.

If I'm analysing lots of data, I tend to write PDF's summarising results. These PDFs can be collated using TeX as I require reports or beamer presentations. Some knowledge of output allows me to set appropriate ranges so that scales are consistent over results.

Consistency in coding style helps as well.

Don't forget that you can use notebooks to run other notebooks.

These seem like generic motherhood statements, and to a certain extent they are. Your mileage will almost certainly vary depending on your problem(s).