Is it possible to figure out in what file a certain packaged symbol lives in (assuming there is no shadowing between multiple identical symbol names)? I would like to have a function that if, given a context or a symbol from a context, opens up the relevant package file.

With Needs Mathematica automatically loads all symbols in each package file of the given context if directories are set correctly with $Path. It seems though that I cannot access the information from which file Mathematica pulls each symbol. It is an overkill to scan through all directories in $Path and through all files in each directory to find the given symbol. Is there an easier method?

  • 3
    $\begingroup$ I'm pretty sure that this information is not stored: to load a package, Mathematica just opens a stream and reads from it as if it were any other sort of input. Still, good question, +1. $\endgroup$ – Oleksandr R. May 28 '13 at 9:55
  • $\begingroup$ In old versions, this would have been a bit easier to do: one just scans the init.m file associated with each package group (e.g. Algebra` , NumericalMath` , ...) and looks at the DeclarePackage[] lines for the appropriate functions. The new way things are arranged now makes things slightly hairier. $\endgroup$ – J. M. is away May 28 '13 at 10:52
  • $\begingroup$ If you follow the convention of having the context match the file name of the package wouldn't this be almost trivial? Do you have a reason not to follow that convention? $\endgroup$ – Mr.Wizard Aug 12 '13 at 7:43
  • 1
    $\begingroup$ @Mr.Wizard If a given context (via Needs) pulls symbols from different files (as e.g. mentioned in this case), than it is not a matter of convention but a matter of taste in which file the symbol is actually stored. $\endgroup$ – István Zachar Aug 12 '13 at 15:28

My previous answer had heavy shortcomings and errors, so I took a deeper breath and figured out a more robust way.

The problem with FindFile["context`"] is twofold. First, it can only return the first file in a possibly long list of files adding to the same context. Second, it might not work on a context extracted from a symbol because symbol contexts might not be identical to the package name. The package specification in Mathematica is confusing: myPackage` means two different things:

  1. FILENAME: If Get["myPackage`"] (or Needs) is called, Mathematica will interpret myPackage as a file path using FindFile to figure out the source.
  2. CONTEXT: In BeginPackage["myPackage`"] (or Begin) and in myPackage`symbol, mypackage is interpreted as a namespace within the memory, and has nothing to do with files. $Packages and $ContextPath only store these contexts but not package-file-path-specifiers of point 1.

There is no guarantee, that the package name agrees with the context name. Though this is the convention, a package defined at path .../myPackage might not contain the line BeginPackage["myPackage`"] in it. One can easily create artificial cases where a package file at .../package.m (called as Get["package`"]) loads symbols like context`sym.

To be clear, any package you design should have at least one file in it which has the same name as the context it generates. If context and file name do not match, either the file is not found (if context is called by Needs, resulting in Get::noopen error) or context is not created (if filename is called by Needs, resulting in Needs::nocont error). Note however, that in the latter case, the file is nevertheless loaded, regardless of the expected context. The reason I say "at least one file" above is because this file then can call any other package file in the same directory (obviously with different file names) or from other directories.

So a foolproof solution to find all package files could only work if it knows the associations between filenames and contexts. To have that, package calls have to be monitored and the collected data stored. In another answer I presented safeGet that can read a package, collect all the files that are touched by successive package calls from within and at the end removes all contexts and symbols introduced during all the calls. With safeGet it becomes possible to capture the necessary data to identify sources. safeGetSource is a light version of safeGet that only captures context -> file associations but not all files (like init.m-s that don't contain context-creating code).

contextJoin[s:{__String}] := StringReplace[StringJoin[#<>"`"& /@s], "`".. -> "`"]
packageButton[file_String] := Button[FileNameTake@file, NotebookOpen@file, 
   Appearance -> "Palette"];

safeGetSource[pkg_String, arg___] := Module[{
    bp, ep, begin, end, contexts = {}, assoc = {}, all, 
    cStack = {$Context}, cpStack = {$ContextPath}},
   Block[{$Packages = $Packages, $ContextPath = $ContextPath, $Context = $Context},
    bp[ctx_] := bp[ctx, {}];
    bp[ctx_, needed_List] := (
      AppendTo[cStack, $Context];
      AppendTo[cpStack, $ContextPath];
      $ContextPath = DeleteDuplicates@Join[{ctx}, needed, {"System`"}];
      $Packages = DeleteDuplicates@Prepend[$Packages, ctx];
      $Context = ctx;
      assoc = Union[assoc, {$Context -> packageButton@$InputFileName}];
      contexts = Union[contexts, {$Context}];
      Needs /@ needed;
    begin[ctx_] := (
      AppendTo[cStack, $Context];
      $Context = contextJoin@{$Context, ctx};
      assoc = Union[assoc, {$Context -> packageButton@$InputFileName}];
      contexts = Union[contexts, {$Context}];
    ep[] := ({$ContextPath, cpStack} = {Last@cpStack, Most@cpStack};
             {$Context, cStack} = {Last@cStack, Most@cStack};);
    end[] := ({$Context, cStack} = {Last@cStack, Most@cStack};);
    Block[{BeginPackage=bp, EndPackage=ep, Begin=begin, End=end}, Get[pkg, arg]];
    all = # <> "*" & /@ contexts;
    Unprotect /@ all;
    Quiet[Remove /@ all];

assoc = safeGetSource@"OpenCLLink`"
 "CCompilerDriver`" -> "CCompilerDriver.m",
 "CCompilerDriver`CCompilerDriverBase`" -> "CCompilerDriverBase.m",
 "CCompilerDriver`CCompilerDriverBase`Private`" -> "CCompilerDriverBase.m",
 "CCompilerDriver`CCompilerDriverRegistry`" -> "CCompilerDriverRegistry.m",
 "CCompilerDriver`CCompilerDriverRegistry`Private`" -> "CCompilerDriverRegistry.m",
 "CCompilerDriver`GenericCCompiler`" -> "GenericCCompiler.m",
 "CCompilerDriver`GenericCCompiler`Private`" -> "GenericCCompiler.m",
 "CCompilerDriver`IntelCompiler`" -> "IntelCompiler.m",
 "CCompilerDriver`IntelCompilerWindows`" -> "IntelCompilerWindows.m",
 "CCompilerDriver`IntelCompilerWindows`Private`" -> "IntelCompilerWindows.m",
 "CCompilerDriver`Private`" -> "CCompilerDriver.m",
 "CCompilerDriver`System`" -> "System.m",
 "CCompilerDriver`System`Private`" -> "System.m",
 "CCompilerDriver`VisualStudioCompiler`" -> "VisualStudioCompiler.m",
 "CCompilerDriver`VisualStudioCompiler`Private`" -> "VisualStudioCompiler.m",
 "CUDALink`" -> "CUDALink.m",
 "CUDALink`NVCCCompiler`" -> "NVCCCompiler.m",
 "CUDALink`NVCCCompiler`Private`" -> "NVCCCompiler.m",
 "CUDALink`Private`" -> "CUDALink.m",
 "GPUTools`" -> "GPUTools.m",
 "GPUTools`CodeGenerator`" -> "CodeGenerator.m",
 "GPUTools`CodeGenerator`Private`" -> "CodeGenerator.m",
 "GPUTools`Detection`" -> "Detection.m",
 "GPUTools`Detection`Private`" -> "Detection.m",
 "GPUTools`Private`" -> "GPUTools.m",
 "GPUTools`SymbolicGPU`" -> "SymbolicGPU.m",
 "GPUTools`SymbolicGPU`Private`" -> "SymbolicGPU.m",
 "GPUTools`Utilities`" -> "Utilities.m",
 "GPUTools`Utilities`Private`" -> "Utilities.m",
 "LibraryLink`" -> "LibraryLink.m",
 "LibraryLink`Private`" -> "LibraryLink.m",
 "OpenCLLink`" -> "OpenCLLink.m",
 "OpenCLLink`Private`" -> "OpenCLLink.m",
 "SymbolicC`" -> "SymbolicC.m",
 "SymbolicC`Private`" -> "SymbolicC.m"

Querying a symbol can now be done as:

 Context@symbol /. assoc

Examining "PacletManager"`` reveals that the one public context calls quite a few files:

 "PacletManager`" -> "PacletManager.m",
 "PacletManager`Collection`Private`" -> "Collection.m",
 "PacletManager`Documentation`Private`" -> "Documentation.m",
 "PacletManager`Extension`Private`" -> "Extension.m",
 "PacletManager`LayoutDocsCollection`Private`" -> "LayoutDocsCollection.m",
 "PacletManager`Manager`Private`" -> "Manager.m",
 "PacletManager`MemoryCollection`Private`" -> "MemoryCollection.m",
 "PacletManager`Package`" -> "Collection.m",
 "PacletManager`Package`" -> "Documentation.m",
 "PacletManager`Package`" -> "Extension.m",
 "PacletManager`Package`" -> "LayoutDocsCollection.m",
 "PacletManager`Package`" -> "Manager.m",
 "PacletManager`Package`" -> "MemoryCollection.m",
 "PacletManager`Package`" -> "Packer.m",
 "PacletManager`Package`" -> "Paclet.m",
 "PacletManager`Package`" -> "PacletManager.m",
 "PacletManager`Package`" -> "Services.m",
 "PacletManager`Package`" -> "Utils.m",
 "PacletManager`Package`" -> "Zip.m",
 "PacletManager`Packer`Private`" -> "Packer.m",
 "PacletManager`Paclet`Private`" -> "Paclet.m",
 "PacletManager`Private`" -> "PacletManager.m",
 "PacletManager`Services`Private`" -> "Services.m",
 "PacletManager`Utils`Private`" -> "Utils.m",
 "PacletManager`Zip`Private`" -> "Zip.m"

Let's see a less complicated example, with a non-conventional package-context association (made for this purpose). Note, that while the package was called as "MyPackage`", the context added to memory and captured by safeGetSource is "MyContext`":

 "Functions`" -> "Functions.m",
 "Functions`Private`" -> "Functions.m",
 "MyContext`" -> "file.m",
 "Test`" -> "Common.m",
 "Test`" -> "TestA.m",
 "Test`" -> "TestB.m",
 "Test`Private`" -> "TestA.m",
 "Test`Private`" -> "TestB.m"

The dependency tree of the above call, created by safeGet:

Mathematica graphics

  • $\begingroup$ I'll admit that I didn't try to understand how it works, I just tried it on the package I wanted to do this with last time: safeGet["OpenCLLink`"]. Unfortunately it seems to go into an infinite recursion. $\endgroup$ – Szabolcs Feb 11 '14 at 15:53
  • $\begingroup$ @Szabolcs Thanks for taking the time to test it! The previous code was crap, please check new version. $\endgroup$ – István Zachar Feb 17 '14 at 18:47
  • $\begingroup$ In contextJoin (first line in code box), str should be corrected to s, right? $\endgroup$ – Szabolcs Feb 17 '14 at 18:49
  • $\begingroup$ @Szabolcs Sure, thanks, corrected. Made the line shorter to fit the page width but forgot to replace all vars. $\endgroup$ – István Zachar Feb 17 '14 at 18:51
  • $\begingroup$ Would you like to submit a patch to the spelunking package to add this (and related) functionality? (This would be a separate function, but part of the same package.) $\endgroup$ – Szabolcs Feb 17 '14 at 18:55

Just a really quick hack that is nevertheless useful sometimes:


If applicable, this opens the package relating to the symbol´s context and primes the search (use F3 to search for subsequent locations of the symbol).

  • $\begingroup$ Actually, SystemOpen uses FindFile under the hood, so that might be more straightforward to use. My problem with this approach is that it can only point you to the first file loaded in the chain, but there is no way (without actually running Get) to know what other files are related. +1 nevertheless. $\endgroup$ – István Zachar Feb 10 '14 at 10:54
  • $\begingroup$ @IstvánZachar you are absolutely correct. This is just something I cobbled together some time ago on the fly, but it has proven useful mainly for packages of my own doing. $\endgroup$ – Yves Klett Feb 10 '14 at 11:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.