There was a recent question asking about how to design modular packages:
This is an important issue for which unfortunately there is no One True Solution. People use various approaches, some better than others. Please check the answers to the linked question.
My question ties into this, but is more specific:
- What is the best way to design reusable sub-modules?
Most packages are meant for end-users. Some other packages are useful for developing other packages built on top of them. One example is Kuba's recently proposed error handling library. Another is my LTemplate package. Both of these are useful only for building other packages on top of them.
Suppose I want to use an error handling library in my own package. How do I deal with this dependency? Do I instruct the users of my own package to also install this other library? This is a sure recipe for failure: Some people will certainly mess up the installation. If they don't, the version of the error handling library they install may not be compatible with my package. But this incompatible version may be required by a different package they are also using ...
I believe that the only robust solution is to bundle the error handling library (the sub-module) with my package in such a way that it won't conflict with any other installations. This question is about how to design the sub-module to make this bundling easy and possible. The eventual goal is to develop a standard for doing this. The standard will hopefully be flexible enough that the design of the sub-module won't constrain the design of the main package too much.
I did try to design LTemplate this way, and it can be used as a case study to identify the problems and requirements when using such sub-modules, and to come up with a better design. This is how LTemplate currently works:
It can be loaded as a standalone package. This is important so that it can be used interactively while prototyping a new package. In this case, it lives in the
LTemplate`
context.It can be embedded as a sub-module of a main package, say,
MyPack
. In this case, it lives in the contextMyPack`LTemplate`
so that it wouldn't conflict with the global installation (LTemplate`
) or with another embedded instance (MyOtherPack`LTemplate`
). Note that these may be different versions, or may be configured differently.Embedding is achieved by placing the complete
LTemplate
directory within theMyPack
package directory, and addingGet["`LTemplate`LTemplatePrivate`"]
afterBeginPackage["MyPack`"]
. This resolves to theMyPack/LTemplate/LTemplatePrivate.m
file, even ifMyPack
lives in a paclet. See an example here.A usual package has the following structure:
BeginPackage
, usages/declarations,Begin["`Private`"]
, definitions,End[]; EndPackage[]
. LTemplate puts the usage/declaration and theBegin
/End
part in a separate file calledLTemplateInner.m
, which is loaded byGet
in several "wrapper" files containing different context setups. LTemplate is normally loaded through one of these wrappers. TheLTemplate.m
wrapper is for standalone loading, and places symbols into theLTemplate`
context. TheLTemplatePrivate.m
wrapper is for loading when LTemplate is embedded in another package, and will set up the contexts so everything is inMyPack`LTemplate`
.Finally, practical needs made it necessary to allow for "configuring" LTemplate by the main package that embeds it. Thus it is also required for it to evaluate a
ConfigureLTemplate
function at loading time to set certain options.
As you can see, all this is pretty complicated, and it is not clear to me that it would play well with some of the possibilities for modularizing packages (e.g. the new style packages that Leonid described). It is also not clear that it is the best solution for a complex, multi-module package, where several of the modules may depend on LTemplate.
The goal of this question is to come up with a design (or standard) that is as simple as possible, can be used as a guide by anyone to create reusable modules, and will play well with the best suggested designs for modular packages.
LTemplate
to have special branch for embedding, does not look scaleable for nested dependencies, as is. $\endgroup$ – Leonid Shifrin Jul 4 '18 at 20:44