Find names of the functions defined in a file

Lets say I have a file named file.m that contains:

test[] := (
Print["test"]
)


How would I go about extracting the function declarations in such a file consistently as a list?

I have figured out how to extract the data without having it execute.

Import["file.m", "Text"]


This is my current code.

Cases[
FullForm[
],
SetDelayed[x__, y__] -> f[x, y]
]

-
Does ToHeldExpression@Import["file.m", "Text"] satisfy your needs? You can get rid of the Nulls in a post-processing step. –  The Toad Jul 1 '13 at 16:48
@rm-rf How do I insure I only get function declarations for example lets say the file has a Print["test"] at the end of the file? –  William Jul 1 '13 at 16:52
I'm not sure that's an easy task or if it can be done in a foolproof manner, given the myriad of ways in which a function can be "defined" in Mathematica... –  The Toad Jul 1 '13 at 16:56
@rm-rf Okay lets just assume a convention is followed and all functions are declared as := (SetDelayed) –  William Jul 1 '13 at 17:00
Well, then why not do something like you did in your question? ;) Cases[ToHeldExpression@Import["~/file.m", "Text"], x_SetDelayed :> Hold@x] (use Unevaluated or Defer or Hold as per your needs) –  The Toad Jul 1 '13 at 17:03

Preamble

This is not such an easy task actually, if you want to do this fast and clean. I have been developing some functionality for one of my side projects, for which I needed to analyze symbols inside packages, so I will share some of the code I ended up with.

Speed

The following function will be two orders of magnitude faster than the one based on Import[...,"HeldExpressions"]:

ClearAll[loadFile];
DeleteCases[
ToExpression[
],
Null
]


hopefully this function is still robust enough.

Benchmarks

Benchmarks on a medium-size package:

file = FileNameJoin[{$InstallationDirectory,"SystemFiles","Links","JLink", "JLink.m"}]; Do[Import[file, "HeldExpressions"], {100}] // AbsoluteTiming Do[loadFile[file], {100}] // AbsoluteTiming (* {4.351563, Null} {0.153320, Null} *)  The format of the result of loadFile is a little different - it just wraps all code in one HoldComplete. But it is easy to transform to the partial HoldComplete wrapped around the pieces. The speed difference can be quite important (for my application it was. I would have been in trouble using Import). Namespace pollution The problem Perhaps even worse than the speed issue is the one of namespace pollution, which is a flaw shared by both methods described so far. The new symbols are being created as a result of parsing the code, and they are created in whatever happens to be the current working context. For example, for the case at hand: Names["Global**"]//Short (* {GlobalPrivatearg,GlobalPrivatearg$,GlobalPrivatee,
<<21>>,GlobalPrivateval,GlobalPrivateval$,GlobalPackage$jlinkDir}
*)


Sugegsted solution

With the implementation details described at the bottom of the post, here is the way I ended up doing this: I introduced the macro with the following signature:

withHeldCodeInTemporaryContext[{var_Symbol, code_}, fname_, opts : OptionsPattern[]]


which is supposed to read in the file fname, parse it into some temporary context, assign the result to a variable var, then execute the code code, and finally clean up that temporary context. Here is an example of use (you will need to run the code posted below for it to work):

Block[
{
heldCode,
file=
FileNameJoin[
{$InstallationDirectory,"SystemFiles","Links","JLink","JLink.m"} ] }, withHeldCodeInTemporaryContext[ { heldCode, Union[ Cases[heldCode,s_Symbol:>ToString[Unevaluated[s]],\[Infinity],Heads->True] ] }, file ] ]  This code collects all the symbols which build up the code being read. The result looks like: (* {And,AppendTo,BeginPackage,Blank,Check,Close, <<78>>,$ContextPath,$Failed,$Input,$Off,$SystemID,$VersionNumber} *)  but one can check that the working context (or any other context) was not polluted. Implementation Here is the code (formatting done using the code formatter palette): SetAttributes[CleanUp,HoldAll]; CleanUp[expr_,cleanup_]:= Module[{exprFn,result,abort=False,rethrow=True,seq}, exprFn[]:= expr; result= CheckAbort[ Catch[Catch[result=exprFn[];rethrow=False;result],_,seq[##1]&], abort=True ]; cleanup; If[abort,Abort[]]; If[rethrow,Throw[result/. seq->Sequence]]; result ] SetAttributes[parseInContext,HoldFirst]; Options[parseInContext]= { LocalizingContext->"MyLocalizingContext", DefaultImportedContexts:>{}, ExtraImportedContexts:>{} }; parseInContext[code_,opts:OptionsPattern[]]:= Module[ { result, context=OptionValue[LocalizingContext], defcontexts=OptionValue[DefaultImportedContexts], extraContexts=OptionValue[ExtraImportedContexts], allContexts }, allContexts={Sequence@@defcontexts,Sequence@@extraContexts}; Block[{$ContextPath},
CleanUp[
BeginPackage[context];Needs/@allContexts;result=code,
EndPackage[]
];
result
]
];

ClearAll[inPrivateContext];
SetAttributes[inPrivateContext,HoldAll];
inPrivateContext[code_]:=
CleanUp[Begin["Private"];code,End[]];

ClearAll[parseInPrivateSubcontext];
SetAttributes[parseInPrivateSubcontext,HoldFirst];
parseInPrivateSubcontext[code_,opts:OptionsPattern[]]:=
parseInContext[inPrivateContext[code],opts];

ClearAll[withTemporaryContext];
SetAttributes[withTemporaryContext,HoldRest];
withTemporaryContext[context_String,{contVar_Symbol,code_}]:=
Block[{contVar=context},
With[{names=context<>"Private*",remove=If[Names[#1]=!={},Remove[#1]]&},
CleanUp[remove[names];code,remove[names]]
]
];

ClearAll[withHeldCodeInTemporaryContext];
Options[withHeldCodeInTemporaryContext]=
{
TemporaryContextName->"TemporaryContext",
ExtraImportedContexts:>{"Global"}
};
SetAttributes[withHeldCodeInTemporaryContext,HoldFirst];
withHeldCodeInTemporaryContext[
{var_Symbol,code_},fname_,opts:OptionsPattern[]
]:=
Module[{tempcont},
Block[{var},
withTemporaryContext[
OptionValue[TemporaryContextName],
{
tempcont,
parseInPrivateSubcontext[
LocalizingContext->tempcont,
ExtraImportedContexts->OptionValue[ExtraImportedContexts]
]
}
]
]
];


The code contains a number of macros,some of which are general (like WReach's CleanUp and a few others), while others specialize the generic ones to the more narrow goals we set here.

-
Well thought out. How does loadFile["testing.m"] pollute the namespace in my example code above? I tried running loadFile["testing.m"] then test[] but I get no output. Am I misunderstanding you or maybe I misunderstand how namespaces work. –  William Jul 1 '13 at 19:05
@Liam In your case, the only non-system symbol you have is test, and you use it anyway. To get the output, you need to use ReleaseHold, since the result of loadFile is code wrapped in HoldComplete. In more complicated cases, where you may have many non-system symbols in your file, all of them will be created in the current working context, unless fully-qualified names were used for them (which is not typical). Generally, such namespace pollution is not what one wants. –  Leonid Shifrin Jul 1 '13 at 19:11
How might you extract the functions from notebook .nb file? –  William Jul 10 '13 at 19:30
@Liam I looked through your pastebin code, but it is still not clear to me what you want to achieve. Perhaps let me explain how withHeldCodeInTemporaryContext works. The variable heldCode you don't have to assign anything. It gets automatically assigned to the code of the file you read by withHeldCodeInTemporaryContext, parsed in a temporary context, and wrapped in Hold. For example, you could call withHeldCodeInTemporaryContext[{helodCode,Print[heldCode]}, "code.m"] to print the parsed held content of your file. The point is, withHeldCodeInTemporaryContext is a scoping construct.. –  Leonid Shifrin Aug 4 '13 at 15:20
@Liam ... (Block-like), and it allows you to access the parsed code in the code parameter, becuase you can use heldCode (or whatever other name you give to this variable) inside code. In particular, if you execute withHeldCodeInTemporaryContext[{myHeldCode, ReleaseHold@myHeldCode},"code.m"], you will effectively execute the (parsed into a temporary context) code of your package. But if you want to access some of the functions of this file by their short name, this will need more work, since withHeldCodeInTemporaryContext will not easily allow that (but this can be done). –  Leonid Shifrin Aug 4 '13 at 15:24

The "HeldExpressions" element for package import should be useful:

In[4]:= Import["foo.m", "HeldExpressions"]

Out[4]= {HoldComplete[test[] := Print["test"]]}

-
Nice! I had not seen this before. –  The Toad Jul 1 '13 at 17:06
@rm-rf definitely a useful addition to your toolkit. –  rcollyer Jul 1 '13 at 18:18

This answer starts with Brett Champion's excellent idea for importing a .m file, but follows through to produce the list of defined functions according the relaxed criterion of finding only the names of functions defined with SetDelay.

testData = Import[SystemDialogInput["FileOpen", "*.m"], "HeldExpressions"];
Union @ Cases[testData, HoldPattern[SetDelayed[f_[___], _]] -> f, ∞]


The reason for using Union is to suppress multiple occurrences of the names of functions with multiple definitions.

To test this, I evaluated the first expression, and when the file open dialog appeared I chose Turtle.m, a turtle graphics package I wrote some years ago. The evaluation resulted in testData being bound to a list of the package's expressions, all wrapped with HoldComplete. Evaluating the second expression gave

{
bk, distXY, extend, fd, goXY, heading, home, isPenDown, isPenUp,
isTurtle, isValid, lf, new, pd, place, pu, rt, setH, setPen, setXY,
toward, track, trackFilter
}


This is a complete list of the turtle graphics functions defined in Turtle.m.

-