19
$\begingroup$

The Package` context contains the following:

enter image description here

What is the use of these undocumented functions?

I am looking for a concise description of the syntax and purpose of the most prominent of these functions, accompanied by examples.

I believe these are primarily used for the auto-loading mechanism of built-in symbols. For example,

enter image description here

There are some references to these symbols within the Parallel Tools. Spelunking reference: SystemOpen@DirectoryName[FindFile["Parallel`"], 2].

The Paclet Manager may also use these to set up auto-loading of paclets.

This mechanism seems to be distinct from DeclarePackage. DeclarePackage is rather rigid and doesn't make it easy to set up flexible auto-loading because it always executes Needs, which will not call Get twice for the same package. I see no easy way to transition an existing single-context package to delayed loading by using only DeclarePackage.

I think it would be useful to spelunk the symbols in the Package context and produce some basic description for them, as well as basic instructions on how to set up auto-loading for packages.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
8
+500
$\begingroup$

CreatePackageCache / AutoloadPackage

CreatePackageCache is called as

Package`CreatePackageCache[
 pkgContext,
 pkgDir,
 Package`AutloadPackage->boolean]

Where pkgContext is a package context declared in the new-style (so that it's possible to find via PackageInformation).

What this does is dumps what is presumably the package data to a .mx file in the pkgDir. If AutoloadPackage is specified it seems to load the package before dumping.

DeclareLoad / ActivateLoad / HiddenImport / ExportedContexts

Package`DeclareLoad is called as 

Package`DeclareLoad[
 {symbols},
 context,
 Package`HiddenImport->boolean,
 Path->pathspec,
 Package`ExportedContexts->{contexts}]

it configures autoloading by setting

Package`ActivateLoad[
 symbol,
 {symbols},
 context,
 {Package`HiddenImport->boolean, Path->pathspec}
 ]

on each symbol in symbols, which in turn clears the OwnValues (and maybe the other *Values) of all the symbols and then loads the context by a Get like mechanism, searching in Path. Note that Package`ActivateLoad can be called on its own.

Still working out what happens to Package`ExportedContexts...

PackageInformation / LoadPackage / ForceLoad

Package`PackageInformation is used to get package information for packages declared in the new style. Package`ForceLoad is used by Package`PackageInformation to specify whether to load the package or not.

Package`LoadPackage loads these new-style packages, but not old-style one.

Spelunking Updates

Long Form

Update 5

PackageImport, PackageScope, and PackageExport all live in the PackageDirectives context which is presumable initialized by LoadPackage

Update 4

Figured out how to use CreatePackageCache based on knowing there are new-style declarations. Note that there must be PackageInformation for pkgContext otherwise your kernel will crash (or at least it has for me).

One oddity is that even though it creates the .mx file, it doesn't seem to be exporting the definitions for my symbols... Need to check this out.

Update 3

There is another symbol, PackageImport which works like PackageExport except it imports the package. One thing worth noting about these new-style packages is that it seems their init files always call BeginPackage and EndPackage still.

Update 2

By running PackageInformation/@Contexts[] I found that there is a new package declaration style as visible in the Macros`, Iconize`, and TextSearch` packages.

I made my a package stub in this style:

Package["TestPKG`"]

PackageScope["blumpf"]
PackageExport["Blah"]
PackageExport["Blah2"]

And by saving this so somewhere included in $Path I could use PackageInformation on this too (although I had to allow ForceLoad. It also works with LoadPackage in a way that packages declared in the old style do not.

PackageExport puts the symbol into toplevel usage. PackageScope is the new Private. We can find more symbols like this by searching for Names["*`PackageScope`*"] after running Get/@Contexts[]

Need to do more searching for the symbols, but I think I'll start in these contexts.

Update 1

Package`DeclareLoad seems to be the main usage function. It takes a set of symbols that are prepped to autoload a set of packages. The loads can be "hidden" via Package`HiddenImport which just means the contexts don't get added to the $ContextPath it seems.

It can also take extra contexts to export via Package`ExportedContexts and a path to look for the package on (default Automatic).

This just configures Package`ActivateLoad in the OwnValues of the symbols, which simply acts as a fancy Get mechanism.

It looks like:

Package`ActivateLoad[loadMyTools,
 {loadMyTools},
 "MyTools`",
 {Package`HiddenImport -> True, Path -> Automatic}]

for example. The second argument are all the other symbols declared in the same DeclareLoad the third is the set of packages, etc. Pretty clear.

Here are the places I've searched so far, so that others can look through them too:

$InstallationDirectory/SystemFiles/Components/MXNetLink/*.m
$InstallationDirectory/SystemFiles/Components/NeuralNetworks/*.m
$InstallationDirectory/SystemFiles/Components/Iconize/*.m
$InstallationDirectory/SystemFiles/Components/Macros/*.m
$InstallationDirectory/AddOns/Applications/Parallel/Kernel/sysload.m
$InstallationDirectory/SystemFiles/Autoload/PacletManager/Kernel/Documentation.m
$InstallationDirectory/SystemFiles/Autoload/PacletManager/Kernel/Manager.m 
$InstallationDirectory/SystemFiles/Kernel/SystemResources/MacOSX-x86-64/sysinit.m
Improve this answer
$\endgroup$
7
  • $\begingroup$ In fact I did give pointers to where to spelunk for internal usages of these functions, mentioning both the parallel tools and the paclet manager. Thus I am aware of what you described in your original post. The reason why I set the bounty was that I hoped that someone would do the spelunking and write a concise, easy to understand answer, that shows how to use the most important functions and gives concrete usage examples. I'd like to have an answer which is easy to read for a large audience and doesn't go into the intricacies of spelunking. Can you please update yours to read like that? $\endgroup$
    – Szabolcs
    Commented Dec 19, 2016 at 8:55
  • 1
    $\begingroup$ Sure thing, I'll pare it down to the basics $\endgroup$
    – b3m2a1
    Commented Dec 19, 2016 at 12:46
  • $\begingroup$ @MB1965 Very interesting, but what are packages in the new style? Is it only that there is a file PacletInformation.m? $\endgroup$
    – Fred Simons
    Commented Mar 27, 2017 at 19:21
  • 1
    $\begingroup$ @FredSimons the PacletInformation.m simply specifies that it's a Mathematica application. Packages in the old style would use BeginPackage and EndPackage. In the new style we declare a package with Package and use the Package* family of functions to define it. Then LoadPackage can be used to load such a package. Check out $InstallationDirectory/SystemFiles/Components/Macros/*.m to see examples of this. $\endgroup$
    – b3m2a1
    Commented Mar 27, 2017 at 19:24
  • 1
    $\begingroup$ @MB1965. Thanks! I am highly surprised. Never seen a package not using BeginPackage - EndPackage. The guide Package Development does not mention the new style. So far, it seems to be completely undocumented. Do you think this new style is meant to replace the old style sooner or later? $\endgroup$
    – Fred Simons
    Commented Mar 28, 2017 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.