# How do you set up private contexts shared between sub-packages?

I have a fairly large project that logically splits into several subunits, and for reasons of testing, documentation, and discoverability, I would like to make each subunit its own (sub-)package, like so:

PillsyPackage
|
+- SubpackageA
|
+- SubpackageB


However, all of these sub-packages need to share some private functions (they all work on the same data structures which should be opaque, so I want the accessors to be private).

Ideally, I'd like to give the sub-packages access to PillsyPackagePrivate. What's the best way to do this in terms of syntax and file layout?

• "need to share some private functions" - in the Standard Add-on Packages, this was solved by setting up a Common.m package containing the private functions that will be shared among other packages that need those functions. – J. M. will be back soon Oct 1 '18 at 18:05
• Use new-style packages. Description here. – Leonid Shifrin Oct 1 '18 at 18:40
• The drawback with applying a Common.m solution is that JITL (Just-in-time-loading via DeclarePackage) doesn't support a "names file". The drawback with a new-style package is that it is not scalable in terms of supporting JITL. If these two aspects are important (and they aren't always) but I argue they increase in importance with the size of a system so they might be for a "fairly large project". I am putting out shortly an extension of BeginPackage-EndPackage (which I will link to) that addresses this very issue. – Ronald Monson Oct 1 '18 at 20:07
• @RonaldMonson DeclarePackage is an old style of doing this, which is no longer used, at least not internally in the company. The current mechanism is to use the paclet symbol autoloading and paclet project loading, which plays well with new-style packages. – Leonid Shifrin Oct 1 '18 at 20:31
• @LeonidShifrin Yes I suspected as much as checking a few weeks ago its vestiges seem to be only in Mma's authortools. But I wonder if something important has been lost here - at least I am yet to see how the Stub attribute transfers as previously discussed ( I'm not just talking about public functions but private ones the OP mentions e.g. what if you want these privately shared functions to incorporate JITL? can the current mechanism handle this?). My sense is that this new mechanism is tilted towards public functions. Naturally, I don't fully understand paclet symbol autoloading so ... – Ronald Monson Oct 1 '18 at 21:07

Here's how I do this. I have a standard load script I build from template for all of my packages. You can see one for my main development package here.

What it does then is look for a directory called Packages in the top-level package directory (although there are config options to change this, if I remember correctly).

Then if the directory structure looks like:

MyPackage
Packages
+ pkg1.m
+ pkg2.m
A
+ subpkg1.m
B
+ subpkg2.m


I'll get out three contexts, "MyPackage", "MyPackageA", "MyPackageB". The way I then make all the symbols in each see eachother when they load is that I use lazy-loading. So first I scan through all the found package files and register all the symbol names in the appropriate context and setup an OwnValues pattern so that when they're called for the first time they will load the package they're declared in using a special PackageGet function. This is done by the PackageDeclare function in the load script.

That PackageGet function then uses InternalWithLocalSettings and SystemPrivateNewContextPath and SystemPrivateRestoreContextPath to set up a \$ContextPath that looks like:

{
"MyPackage",
"MyPackageA",
"MyPackageB",
"MyPackagePackageScopePrivate" (* I put a bunch of utility functions here *)
}
`

This way each symbol sees each other when they load, but no private declarations or anything like that.

I haven't fully described what I do, as I also have ways to handle circular dependencies and things, but this is a basic overview that perhaps one might implement for themselves.

I have found it to be a useful and powerful idiom reminiscent of the way python handles package loading.