I'm relatively new to git — currently overseeing an offshore .NET-based development project on GitHub within a private business account, but little experience beyond that.

I'd like to set up repos for storing my Mathematica projects (including one for my dissertation work). Initial tests revealed that git will make an unusable mess out of .nb files. Searching lead me to resources mentioning that notebooks contain too much metadata to diff/merge cleanly (without some specialized logic).


What git customizations are appropriate when Mathematica files and projects are involved (i.e., Workbench projects)? I'm a git noob, so be as explicit as possible with any instructions/advice! Are there any specific considerations to be made when using GitHub to host my Mathematica projects?

Resources that might help:

  1. Pro Git Book:

  2. Are there suitable versioning systems for Mathematica notebooks?

  • 1
    $\begingroup$ I think there is a way to make .nb files be treated as binary files. Even the smallest change makes the whole .nb file "dirty" Maybe you could do this for .nb files, .mx files etc. and have .m files be treated normally. I don't think this is in .gitignore, though… will look and see if I can find the settings. $\endgroup$
    – Eric Brown
    May 31, 2013 at 1:02
  • $\begingroup$ I made some edits to your question — GitHub is just a service that uses git, which is the underlying VCS technology. Your questions are all about using git and not GitHub. Think of GitHub as being to your data/source as what Facebook/Flickr are to your photos. Just a place to store and share something and "like"/"follow" others (except, without the FB creepiness). There is nothing Mathematica specific about using GitHub. Secondly, the title mentioned .gitignore, while the body asked about other stuff, which was a disconnect... Looks like you aren't after just the .gitignore afterall $\endgroup$
    – rm -rf
    May 31, 2013 at 1:16
  • 1
    $\begingroup$ I think he has a valid Mathematica ecosystem question. It would probably be a disaster to merge two .nb files, but it would seem appropriate to merge .m files. I think everything else (wdx, mx) is a binary. If nb could be treated as binary, then I think everything would be fine. $\endgroup$
    – Eric Brown
    May 31, 2013 at 1:40
  • $\begingroup$ @rm-rf - I understand all the Git vs. GitHub stuff... I explicitly mentioned both git and GitHub in the question because while git may be the underlying vcs, GitHub is bound to be a more likely search term. $\endgroup$ May 31, 2013 at 3:02
  • 1
    $\begingroup$ @EricBrown - Yes, I think the treat .nb as binary is one tweak MMA users should be using---but I'm not exactly sure how/where it should be used. And are there consequences for how this and other settings are applied for team/public projects (with multiple participants)? $\endgroup$ May 31, 2013 at 3:07

4 Answers 4



Using git to version control your Mathematica projects is a good choice and you will not regret it. However, like with most tools, it has its own learning curve, the difficulty of which will depend on how comfortable you are with using unix style command line tools. While the basics of git are easy to learn and use (especially if you're a single user, using it to simply "save state"), it takes more ninja-fu to do more complicated tasks when working in a team environment (such as resolving merge conflicts, rebasing, rewriting history, etc.).

In this answer, I will not go over the details of using git — the Git – Book that you linked to is an excellent resource and it is a waste of time to regurgitate that. Instead, I'll try to focus on certain aspects of git that are useful in solo/collaborative work, some Mathematica specific settings, and also briefly discuss project hosting on GitHub. Most of my answer is based on personal experience, although I'm far from being a git ninja.

General Mathematica + git advice:

1. Avoid using Notebooks as your primary source code

As you noted in the question, there is a lot of meta data bundled inside each notebook. Although you can turn these off with FileOutlineCache -> False in the options settings, I have never managed to satisfactorily place notebooks under version control. Notebooks can contain In/Out labels, cell grouping rules, modification times, front end version info, etc. some of which can change by simply opening and re-saving without any modifications. In a team environment, this can lead to disasters and you'll be spending all your time resolving merge conflicts. The worst part — you'll be doing this with the ugly internal cell expressions in the notebook!

Strictly speaking, git should be used only for actual source code i.e. version control of code that you/someone else personally wrote/modified and not something that's modified in possibly unknown/undetermined ways by an external program. A one letter change should not result in a 10 line diff output. Placing notebooks under VCS is orthogonal to the goals of a VCS and should be avoided as much as possible. As I mentioned earlier, if you're a single user, working in a single branch all the time and are using git solely as a daily backup, then you could get by with adding notebooks to your repo. Marking a notebook as a binary with gitattributes is another option (and in fact, recommended if you must place it under VCS).

2. Get comfortable with packages

As a continuation from the previous point, if you want to use git effectively, then you should start maintaining your code in Mathematica packages (or plain m files). This does not mean that you should give up the notebook entirely. The notebook still serves as a useful, interactive tool to quickly develop/test/modify functions. Just remember to copy them over to the package when you're done with it, so that they can be placed under version control.

As you start writing more packages and become familiar with programming in Mathematica without the front end, you'll get better at writing your functions and programs directly in the .m file and only using the front end for final testing and improvements.

3. Your development environment is not the same as the deployment environment!

If you create a project in the Workbench, your project structure looks something like this:

   |-- Project/
   |      |-- Project.m
   |      |-- Kernel/
   |             |-- init.m
   |-- Project.nb
   |-- Paclet.info
   |-- Test/
         |-- Test1.mt
         |-- Test2.mt

whereas your deployed environment looks like this

   |-- Kernel/
   |      |-- init.m
   |-- Project.m
   |-- Paclet.info

These two directory and file structures are different, so trying to deploy the development setup directly will give you an error. I must note — by "deploy", I mean copying over to $UserBaseDirectory as in the case of a simple application. For more complicated projects with documentation, you'll have additional complexities, but the concerns remain the same.

When hosting your project on GitHub, you should decide – do you want the repo to be for development purposes only (i.e., you don't expect anyone to pull and install it to $UserBaseDirectory) or do you plan on making it a combined repo which has both the deployed and development branches? If it's the latter, then I suggest maintaining a separate development branch with the first structure and a distinct master branch with the second structure and never let the two mix.

The easiest way to do this would be with an orphan branch (which, as the name suggests, has no parent). Let's say you have your development repo in ~/dev/project and remote repo (up-to-date) at github.com/user, then the following steps will help you establish a separate master

cd ~/path/to/mma/apps/dir/
git clone github.com/user/project.git
git checkout --orphan master
rm -rf ./*

Now in your Workbench project, deploy (and build, if necessary) your project to ~/path/to/mma/apps/dir/project, verify the files and then commit to master. You now have a master branch that is clean and immediately deployable. Make sure to never attempt to merge from the development branch to the master and vice versa. It simply won't work because they have no parents in common (remember, we used an orphan branch).

This means that all your new code/modifications must originate in the development, be versioned and pushed/pulled to the remote(s), from others, etc. and deployed and committed to master when ready (it doesn't make sense to push minor commits to master) only via the Workbench (or a custom deploy script). Note that the Workbench does not preserve file permissions when deploying, so if that is critical to your application, then you must use a custom deploy script. The master branch serves merely as the face of the application to the end user, whereas all the gory history is in the development branch.

General git + team work advice:

1. Use branches!

First, if you're using git, then you already know (or now you do) that branches are light weight (meaning, files are not copied like in SVN) and only contain a reference to the parent commit. This means that all your trials/experimentations/goofs should be done in a temporary branch which can then be merged to the main development branch when you're satisfied. If you're not, simply delete the branch and create a new one. As simple as that! Learn to make extensive use of branches on your local copy, but don't do this on the remote.

In addition to the main development line, it is very helpful if the team members each maintain their own remote branch that is up-to-date at all times, so that others can pull changes from them. This allows one to work in their own "private space" (the branch), while allowing others to publicly view and comment/discuss code changes before pulling. GitHub also allows you to issue pull requests from one branch to the other in the same repo, so that's also a possibility to let someone (say, project manager/team member) know that your changes are ready to be reviewed and merged.

2. Use blame and diff to track down changes

When working in teams, you might find that a certain line of code is causing you trouble and you want to find out who introduced it in which commit, so that you can track that person down and discuss why the change was made and how to work around the present situation. Use git blame and git diff for that.

Although "blame" is a rather strong term (we're not trying to blame anyone; just trying to fix the problem), the blame tool is useful because it annotates each line with the author that last modified it and the commit SHA that introduced the modification. git diff is also needed here to track down what was changed (or perhaps deleted).

3. Squash/rebase your history to keep it clean

This is more of a personal preference for you/your team on how you want to maintain your repo. Some users create a commit for every small change, but it quickly gets boring to see 10 checkins from one user that are variants of

 -- fixed comma
 -- changed indentation
 -- fixed typo
 -- missing ; inserted

While this is fine in a local repo, it is better to send just a single commit upstream. Using git squash, you can "squash" or collapse all of those to a single commit.

Rebasing is a more complicated procedure (but a very useful one) that lets you rewrite history as it didn't happen. For example, suppose feature A logically comes before feature B, but due to the twisted nature of development, was implemented after B. You can use the rebase tool to change the order of the commits. Look up rebasing – it's worth learning.

4. Don't rewrite history on the remote!

This is a very important warning! Every time you change history, you are changing the SHA of the commit. Rebasing, squashing, etc. (and other complicated changes with the git filter) all change the commit SHAs. While this is easily fixed (or not an issue) on local branches, once you've pushed to the remote, you should not attempt to rebase or make any such modifications, as you'll end up breaking your team mates' code (if they've already pulled your changes).

If you must modify the history of the remote commit, do it only if you know no one has pulled yet (e.g., within a minute of pushing). If someone has pulled it and you still need to modify, then let the other person know so that they can work around it. All in all, history modification is a messy operation and try to avoid it in shared code.

This is all I have time for now, but hopefully it provides a broad overview of using git for Mathematica projects and a little bit on using it for team work. There is not much use in focusing on GitHub alone, as the issues you raise (and I've tried to address) are fundamentally about git and not the specific online service you use.

As to what the .gitignore should have — well, just about anything you don't want added to VCS.

  • $\begingroup$ very sound advice, thanks. $\endgroup$
    – Tuku
    May 31, 2013 at 14:51
  • $\begingroup$ Thanks for all the info. I'm already thinking of follow-ups regarding some team workflow examples and how best to contribute ideas/improvements/fixes to projects initiated by others on GitHub, without pi$$ing anyone off. (issues centered around 'what constitutes an improvement' come to mind---Mathematica's flexibility makes this somewhat of a challenging issue, especially given that lack of a standard 'planning tool' for MMA development, and the fact that most (?) of us don't practice test driven development.) I'll give ya the ol'check mark in a couple of days, in case you want to edit more. $\endgroup$ Jun 5, 2013 at 15:14
  • $\begingroup$ @telefunkenvf14 I can edit (later) to address some of that, but the remaining are highly dependent on the project lead and there's not much I can say. For instance, some folks accept pull requests for small modifications, whereas others expect the feature to be fleshed out in detail, some are anal about how the git commit message is written/formatted (Tim Pope is a good example), some base it on the experience/ability of the user (expect pros to contribute more), etc. Sometimes a simple comma might be the source of a huge bug... (contd.) $\endgroup$
    – rm -rf
    Jun 8, 2013 at 16:19
  • $\begingroup$ ... so while it looks like a "minor change" from a diff view, it might be the change that allowed your product to ship in time. Discuss in advance with your project/team lead and other devs as to what constitutes a decent contribution and what things will piss them off. There are some common sense rules as well — for instance, don't go around changing all the indentation and formatting in someone else's repo to your liking or get into a tab vs spaces holy war, etc. If they have bad practices, discuss with them and show an improvement in your fork and confirm before initiating a pull request. $\endgroup$
    – rm -rf
    Jun 8, 2013 at 16:19
  • 3
    $\begingroup$ In summary, Mathematica was not prepared to be in a version control system... Not even the "undo" feature they made it right. $\endgroup$ Aug 20, 2013 at 10:18

The solution I am currently using is to work with *.wls script files.

You can do nearly all the things you do with usual *.nb notebooks files (as shown in my screen shot).

The big advantage is that outputs and graphics are not saved into these *.wls files

(you have to click on Run All Code to regenerate them).

By consequence you can use git as usual (like if you worked with C++ files for instance), your Mathematica files are still small.

Illustration my demo.wls file is:

#!/usr/bin/env wolframscript
(* ::Package:: *)

(* ::Chapter:: *)
(*You can do a lot of things with ".wls" files*)


(* ::Text:: *)
(*Extra information can be found reading the official Mathematica doc: http://reference.wolfram.com/lang

(* ::Subchapter:: *)
(*The big advantage is that graphics and outputs are not saved:*)


(* ::Subsection:: *)
(*So you can use git as usual!*)

and my Mathematica (v11 under Linux) front end is:

enter image description here

You can check that nor the plot, nor the text output is saved. You have to regenerate them. This is perfect for version control systems like git.

  • 1
    $\begingroup$ Can you write a package in .WLS? The WLS link says "If your code is more complex than a plain list of commands, you may want to consider writing a more structured package" which isn't very clear one way or the other. $\endgroup$ Sep 2, 2019 at 18:25
  • 2
    $\begingroup$ @berniethejet for packages, you must use .m or .wl files (AFAIK there is no real difference between these two: community.wolfram.com/groups/-/m/t/1507777). Packages (.m or .wl) files do not store their output (like .wls) hence there is no problem to use git with them. $\endgroup$ Sep 2, 2019 at 20:27

A new possibility is to use mathematica-notebook-filter which parses Mathematica notebooks and strips all output cells and metadata so that these are not committed into the version control system.

In the specific case of git, it is quite easy to integrate mathematica-notebook-filter so that git automatically cleans the output and metadata when calculating diffs through the use of gitattribute filters. You will need to have mathematica-notebook-filter filter installed and added to your path variable (or adapt the configuration below to point to the binary) and add the following line to your ~/.gitattributes file:

*.nb    filter=dropoutput_nb

This instructs git to parse all files matching *.nb with the dropoutput_nb filter which is defined in your ~/.gitconfig as:

[filter "dropoutput_nb"]
    clean = mathematica-notebook-filter
    smudge = cat

If, for some reason, you want to have a specific Mathematica notebook committed with all output and metadata, you can disable the filter in the project's .gitattributes file by adding:

notebook_file.nb    !filter

Disclaimer: I am the author of this tool. It is open source and feedback (both good and bad) is appreciated. Contributions are welcome on Github.

  • 4
    $\begingroup$ (+1) I think this could be v. useful. As someone who does development in notebooks with a heavy usage of initialization cells, the ability to git diff/edit/resolve two conflicting notebooks (via the front-end) would be of great benefit in setting the right modularization granularity within teams using this style. $\endgroup$ Sep 6, 2017 at 6:57

A different use case for git/github is to just show off code (e.g. to hiring managers). I tried Picaud's method of making a .wls file but found every line appeared commented out on github. This really reduced readability, which is important for this use case. (The commenting-out seems to be done by the .wls file, not by github. Any text editor will show the same result.)

A much more readable format is to make a .md file (markdown file, works mostly like a .txt file) that contains

(*copy and paste your code in here *)

This will show up nicely formatted and colored on github (except for Greek letters):

enter image description here

(I got this trick from Mathematica Syntax Coloring in GitHub README)

  • 3
    $\begingroup$ While this is good and (slightly) relevant information, this is about a completely different subject. The question here is about code committed, not about making a README look good. $\endgroup$ May 12, 2021 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.