How do you set up private contexts shared between sub-packages?

Question

I have a fairly large project that logically splits into several subunits, and for reasons of testing, documentation, and discoverability, I would like to make each subunit its own (sub-)package, like so:

PillsyPackage`
  |
  +- `SubpackageA`
  |
  +- `SubpackageB`

However, all of these sub-packages need to share some private functions (they all work on the same data structures which should be opaque, so I want the accessors to be private).

Ideally, I'd like to give the sub-packages access to PillsyPackagePrivate`. What's the best way to do this in terms of syntax and file layout?

Answer 1

Here's how I do this. I have a standard load script I build from template for all of my packages. You can see one for my main development package here.

What it does then is look for a directory called Packages in the top-level package directory (although there are config options to change this, if I remember correctly).

Then if the directory structure looks like:

MyPackage
 Packages
  + pkg1.m
  + pkg2.m
  A
   + subpkg1.m
  B
   + subpkg2.m

I'll get out three contexts, "MyPackage`", "MyPackage`A", "MyPackage`B`". The way I then make all the symbols in each see eachother when they load is that I use lazy-loading. So first I scan through all the found package files and register all the symbol names in the appropriate context and setup an OwnValues pattern so that when they're called for the first time they will load the package they're declared in using a special PackageGet function. This is done by the PackageDeclare function in the load script.

That PackageGet function then uses Internal`WithLocalSettings and System`Private`NewContextPath and System`Private`RestoreContextPath to set up a $ContextPath that looks like:

{
 "MyPackage`",
 "MyPackage`A`",
 "MyPackage`B`",
 "MyPackage`PackageScope`Private`" (* I put a bunch of utility functions here *)
 }

This way each symbol sees each other when they load, but no private declarations or anything like that.

I haven't fully described what I do, as I also have ways to handle circular dependencies and things, but this is a basic overview that perhaps one might implement for themselves.

I have found it to be a useful and powerful idiom reminiscent of the way python handles package loading.