Here I offer the safe version of Get
that can be used successively to collect all the source files and contexts of packages without polluting the memory (too much).
What it does
I have practically reverse-engineered all the necessary functions (Get
, Needs
, BeginPackage
, Begin
, EndPackage
and End
) so that I could inject the monitoring code for introspection. $Context
and $ContextPath
stacks are correctly set and reset, needs
and get
behave exactly as their built-in counterparts (at least in my extensive testing). During execution, all contexts created and files encountered (even if not opened by Needs
being already in $Package
) are collected. At the end, $Context
, $ContextPath
and $Packages
are reset to their original values, and every symbol of every visited context (even private ones) are removed as an attempt to recreate the before-call state of memory as best as possible.
contextJoin[s: {__String}] := StringReplace[StringJoin[#<>"`"& /@s], "`".. -> "`"]
packageButton[file_String] :=
Button[FileNameTake@file, NotebookOpen@file, Appearance -> "Palette"];
safeGet[pkg_String, arg___, opts : OptionsPattern[]] := Module[{
bp, ep, begin, end, get, needs, contexts = {}, files = {},
assoc = {}, edges = {}, all, $input, from = "Global`",
cStack = {$Context}, cpStack = {$ContextPath}},
Block[{$Packages = $Packages, $ContextPath = $ContextPath, $Context = $Context},
Off[General::shdw];
Options[get] = Options@Get; (* TODO: key for encoding... *)
get[package_String, key___String, getopts : OptionsPattern[]] :=
Block[{$Path = $Path, System`Private`$InputFileName},
If[FreeQ[$Path, #], AppendTo[$Path, #]] & /@ OptionValue@Path;
System`Private`$InputFileName = FindFile@package;
AppendTo[edges, from -> System`Private`$InputFileName];
files = Union[files, {System`Private`$InputFileName}];
$input = package;
Block[{from = System`Private`$InputFileName, stream, last, temp},
stream = OpenRead[System`Private`$InputFileName];
While[(temp = Read[stream, Hold@Expression]) =!= EndOfFile,
last = ReleaseHold@(temp /. {HoldPattern@$Input -> $input})];
Close@stream;
last]];
needs[package_String] := needs[package, FindFile@package];
needs[package_String, file_String] := Module[{res},
If[FreeQ[$ContextPath, package], PrependTo[$ContextPath, package]];
(* Sic! Added regardless whether package was really loaded or not.*)
If[FreeQ[$Packages, package],
PrependTo[$Packages, package];
If[FreeQ[files, file], AppendTo[files, file]];
res = get@file;
If[FreeQ[$Packages, package], Message[Needs::nocont, package]];
(* Sic! Only $Packages are checked but not $ContextPath. *)
res
]];
bp[ctx_] := bp[ctx, {}];
bp[ctx_, needed_List] := (
AppendTo[cStack, $Context];
AppendTo[cpStack, $ContextPath];
$ContextPath = DeleteDuplicates@Join[{ctx}, needed, {"System`"}];
(* DeleteDuplicates keeps order, unlike Union. *)
$Packages = DeleteDuplicates@Prepend[$Packages, ctx];
$Context = ctx;
assoc = Union[assoc, {$Context -> packageButton@System`Private`$InputFileName}];
contexts = Union[contexts, {$Context}];
needs /@ needed; (* Public import as it should be for BeginPackage *)
ctx
);
begin[ctx_] := (
AppendTo[cStack, $Context];
$Context = contextJoin@{$Context, ctx};
assoc = Union[assoc, {$Context -> packageButton@System`Private`$InputFileName}];
contexts = Union[contexts, {$Context}];
ctx
);
ep[] := ({$ContextPath, cpStack} = {Last@cpStack, Most@cpStack};
{$Context, cStack} = {Last@cStack, Most@cStack};);
end[] := ({$Context, cStack} = {Last@cStack, Most@cStack};);
Block[{BeginPackage = bp, EndPackage = ep, Begin = begin,
End = end, Needs = needs, Get = get}, Get[pkg, arg]];(* Main call. *)
all = # <> "*" & /@ contexts;
Unprotect /@ all;
Quiet[Remove /@ all];
On[General::shdw];
{
"Package" -> pkg,
"Contexts" -> contexts,
"Files" -> (packageButton /@ files),
"Associations" -> (assoc),
"Names" -> (Names /@ all),
"DependencyGraph" ->
Graph[edges, EdgeStyle -> Arrowheads@Medium,
GraphLayout -> {"LayeredEmbedding",
"RootVertex" -> edges[[1, 1]], LayerSizeFunction -> (.4 &),
"LeafDistance" -> 1},
VertexShapeFunction -> (Inset[Style[packageButton@#2, Black], #1, #3] &),
ImageSize -> 1000]
}
]];
It returns the following things:
- The package name
- All the contexts (even private ones) created during the call
- All the files visited during the call
- An association list of contexts -> files, so that later any symbol (i.e. their source file) can be easily tracked.
- A dependency graph of the related package files.
Let's try it with a complex package call:
safeGet@"OpenCLLink`"
{"Package" -> "OpenCLLink`",
"Contexts" -> {"CCompilerDriver`", "CCompilerDriver`CCompilerDriverBase`",
...
"LibraryLink`", "LibraryLink`Private`", "OpenCLLink`",
"OpenCLLink`Private`", "SymbolicC`", "SymbolicC`Private`"},
"Files" -> {"CCompilerDriverBase.m", ..., "WGLPrivate.m"},
"Names" -> {{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {},
{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {},
{"LibraryLink`$LibraryError"}, {}, {}, {}, {}, {}},
"DependencyGraph" -> Graph[...]}
The graph is the most informative:

Note that all the contexts touched during the call (stored in contexts
) and their symbols are correctly removed, so that at the end, no symbols remain to be listed by Names
. The only exception is LibraryLink`$LibraryError
: it is probably used by some other function/package already loaded.
Another example:
"DependencyGraph" /. safeGet@"PacletManager`"

Some details of the code
- Files read directly with
OpenRead
or Import
(i.e. omitting Get
or Needs
) are not captured!
- Since
$Input
has attribute Locked
, it cannot be modified locally like $InputFileName
, so the Hold
/ReleaseHold
is required to make the replacement structurally by hand. $Input
is used e.g. in the PacletManager`PacletManager.m
file; $InputFileName
is used extensively in many packages (e.g. CUDALink`
).
- Iterative reading is necessary (
ReadList
won't work) because any context called with BeginPackage
must be created before context-symbols are parsed, otherwise new symbols would be created in the Global`
context.
- A list of non-safe functions can be suppressed to minimize the footprint the package makes by adding them to the replacement list where
$Input
is replaced inside get
(e.g. Print|Save|... -> Hold
).
- Options
Method
and CharacterEncoding
of Get
are not implemented, but I did not find any case where they are actually used.
- No error messages when an invalid context is called (
BeginPackage::cxt
and Begin::cxt
).
- Symbols created (and protected) by the package directly in the
Global`
context
during the call cannot be recreated again in a second call, so the standar error will be generated Mathematica not being able to overwrite the symbol (Set::write
and SetDelayed::write
). These symbols could perhaps be captured and suppressed via the
$NewSymbol
.