# Import package with correct symbol contexts

I am looking to do some automated analysis on packages, e.g. automatically check for common mistakes. Mathematica makes it relatively easy to manipulate code as data, and Import can get the expressions from a package, so I was hoping that this would be possible.

The biggest obstacle so far is that having the proper context for each symbol is often critical, and Import doesn't handle this.

Is there an existing solution that can infer the correct context of each symbol when importing a package?

Example:

code = "
BeginPackage[\"Foo\"]
MyFoo::usage = \"MyFoo[] is a function\";
Begin[\"Private\"]

foo[] := 3 (* helper function *)
MyFoo[] := foo[]^2 (* my function *)

End[]
EndPackage[]
";

ImportString[code, "HeldExpressions"]
{HoldComplete[BeginPackage["Foo"]],
HoldComplete[MyFoo::usage = "MyFoo[] is a function";],
HoldComplete[Begin["Private"]],
HoldComplete[foo[] := 3],
HoldComplete[MyFoo[] := foo[]^2],
HoldComplete[End[]],
HoldComplete[EndPackage[]]}

I want HoldComplete[FooMyFoo::usage = "MyFoo[] is a function";], HoldComplete[FooPrivatefoo[] := 3], etc.

I know that creating something that always works is plainly impossible without rewriting the evaluator itself. What I am hoping for is something that works well enough to be practically useful and handles most common package structures. And something that hopefully already exists.

• If you want your analysis to be completely static, you need to have all the source files for dependencies and their dependencies available as well. Is that the case for your indented use cases of this? Aug 23, 2016 at 18:00
• @Leonid Yes, but as a start I would be content with a single-file solution. Also, the analysis doesn't need to be completely static in the sense that if there is a way to record the expressions from the package while the package is being loaded, that is fine, even preferable. But I don't think it's possible. Is there some undocumented way to hook into the parser? Also, I realize that such a tool must to some extent be influenced by the current environment and what symbols already exists in contexts in the context path. Aug 24, 2016 at 7:57
• @Szabolcs so at the end it can load the package except of importing it?
– Kuba
Aug 24, 2016 at 8:08

code = "
BeginPackage[\"Foo\"]
MyFoo::usage = \"MyFoo[] is a function\";
Begin[\"Private\"]

foo[] := 3 (* helper function *)
MyFoo[] := foo[]^2 (* my function *)
bar[]:=DynamicModule[{x},{Dynamic[x],x}];

End[]
EndPackage[]
"

Added a line with DynamicModule to show obstacles.

1. ### Printing with full names:

• As Boxes

We can use FullForm with context for each symbol? to show contexts in read expressions representations.

But it will not prevent expressions with FormatValues (MakeBoxes upvalues) from being applied, e.g.

Block[{$ContextPath = {},$Context = "System"},
RawBoxes@ToBoxes@Dynamic[x]
]

x  (*DynamicBox[...]*)

There is DynamicBox inside but one would expect Dynamic[Globalx] anyway.

So let's use code from Unchanged Box form of an arbitrary expression

ToInputFormBoxes = MathLinkCallFrontEnd[
FrontEndUndocumentedTestFEParserPacket[#, True]
][[1, 1]] &;

withContext = Block[{$ContextPath = {},$Context = "System"},
RawBoxes @ ToInputFormBoxes @ ToString[ #, InputForm ]
] &;

withContext[Dynamic[x]]

Dynamic[Globalx]

• As a String

Alternatively, one just use ToString and not care but I prefer boxes as syntax highlighting is respected, etc.

Block[{$ContextPath = {},$Context = "System"},
ToString @ Dynamic[x]
]

"Dynamic[Globalx]"

2. ### Reading package step by step

Now we are ready we can read expression by expression and return formatted output. The key is an ability of Read to do Read[..., Hold[Expression]].

This example is created for a string but StringToStream can be replaced with OpenRead.

str = StringToStream[code];
list = Reap @ While[
expr =!= EndOfFile
,
Sow[ Interpretation @@ {withContext[expr], expr} ];
ReleaseHold[expr];
];
Close[str];
list[[-1, 1]]
]