4
$\begingroup$

I have a code that is made up of 30+ .m files. of this, about 20 are just files with generic definitions, and 10 are files with a set of routines that define a set of calculations to be made in series. As the project grows, I start to have issues with names of variables. I would like to separate the various sets of calculations definide in each of thiese files, while still keeping the code all together. My idea would be to turn this code into a package. I would need

  1. The package to be able to read some variables in the global context, that I would define in the initialization notebook before loading the package, with something like
variable1=1;
variable2=2;
<<package

These variables would be used as intialization variables, that will tell the package what to load when it comes to load certain things, like

If[variable1==1, 
function[x_]:=function1[x];,
function[x_]:=function2[x];
];
  1. On load, the package initialization starts by defining in the package context (like: MyPackage') a function that loads the 20 files with definitions, that from now on I will call libraries. This function should be able to be accesses from anywhere inside the package.
  2. It then loads each of the 10 files with routines, that form now on I call "modules" each in a separate namespace, like MyPackage'Module1'. In each namespace, before loading the module, it calls the function defined in (2), so that each namespace contains a full copy of all the shared libraries, so something like
BeginNamaspace MyPackage'  
LoadLibs:=(
Get["lib1.m"];
Get["lib2.m"];
);
BeginNamaspace MyPackage'Module1'
LoadLibs;
Get["module1.m"];
  1. A very limited amout of commands/functions should be able to be shared between the various contexts, but I would like the different context to remain as isolated as possible, so that if I happen to define a variable a in a module and a variable a in another module, this would not cause any coflict.
  2. An even more limited amout of commands should be made public, i.e. visible to the end user, so that he cannot mess up with anything internal

Now, I have no experience at all with packages, every time I tried to code anything mroe complex than an "hello world" package, I have been running into some issue, usually related to context handling: variables that are unable to be found when they need to be found, or variables that get shared/global when they are not supposed to, and so they get overwritten by other parts of code that also use them. So I am here to ask for the easiest way to implement this. Note: I'd like to keep a version of my code that in non-package version, i.e. like it is now. So I am possibly looking for an implementation where I can just copy/past all these files in 1 or 2 folders (maybe one for the shared libraries and one for the modules?), and then write some loading files that take care of what to load and in what context. So far I tried both the standard way to make packages and the new way without "beginpackage", but I have run into trouble in both of them. One of my principal issues is that it seems that if I beclare a namespace - by either beginpackage, or Package, and then use Get or << to load a file, the content of the file is loaded in Global, that is something I do NOT want.

EDIT: based on the comment-s suggestion, I have tried to build an example of code. As I got a problem, obviously this example is not working, but I hope it gives an Idea of what I want to do. I am trying to use the old package format - that is easier to understand. I tried to do something similar also with the new package format, but also in that case it did not work.

Directory structure

ExampleOLD
+ExampleOLD
 +Kernel
  +init.m
 +libs
  +lib1.m
  +privatelib1.m
  +settings.m
 +modules
  +module1.m
  +module2.m
 +exampleOLD.m
+exampleOLD.nb
+PackletInfo.m

file content: init.m

$rootdir=FileNameJoin[Drop[FileNameSplit[$InputFileName], -2]];
SetDirectory[$rootdir];
$libdir=FileNameJoin[{$rootdir,"libs"}];
$moddir=FileNameJoin[{$rootdir,"modules"}];
Get["exampleOLD`exampleOLD`"];

lib1.m

function1[x_]:=x+1; (*a very important function used all around the package*)
function2[x_]:=x-1; (*a very important function used all around the package*)

privatelib1.m

If[$setting1==1,
function3[x_]:=x+1;, (*a very important function used only in module1*)
function3[x_]:=x-1; (*a very important function used only in module1*)
];

settings.m

If[ValueQ[Global`$setting1], (*checks if user has defined this variable before initialising the package*)
    $setting1=Global`$setting1; (*if yes it loads the value form global in the context where this file is loaded*)
    $setting1=1;    (*if not it uses a default value*)
];
If[ValueQ[Global`$setting2], (*same for setting 2*)
    $setting2=Global`$setting2; (*sidenote: I could remove the vbalues in global after loading in the package*)
    $setting2=1;
];
$setting3=3;
$setting4=4;

module1.m

BeginPackage["exampleOLD`module1`"]
(* Exported symbols added here with SymbolName::usage *) 


Begin["`Private`"]
(* Implementation of the package *)

LoadSettings; (*shoudl load all values in THIS context*)
SetDirectory[$libdir];
<<privatelib1.m; (*same*)
SetDirectory[$rootdir];
g[x_]:=ftoshare[x]-1+a; (*this function needs module 2 do share some symbols*)

End[]

EndPackage[]

module2.m

BeginPackage["exampleOLD`module2`"]
(* Exported symbols added here with SymbolName::usage *) 


Begin["`Private`"]
(* Implementation of the package *)

LoadSettings; (*shoudl load all values in THIS context*)
SetDirectory[$libdir];
<<privatelib1.m; (*same*)
SetDirectory[$rootdir];
ftoshare[x_]:=x^2+1; (*I would like to share this function with module1*)
a=ftoshare[$setting3]; (*I would like to share this variable with module1*)

End[]

EndPackage[]

exampleOLD.m

BeginPackage["exampleOLD`"]
(* Exported symbols added here with SymbolName::usage *) 


Begin["`Private`"]
(* Implementation of the package *)

$rootdir=Global`$rootdir; (*loads the value form global into the package context. Should be not accesible from outside, only within the  package. Does not work either here, nor in the non-private section.*)
$libdir=Global`$libdir; (*same*)
$moddir=Global`$moddir; (*same*)

LoadSettings:=(
SetDirectory[$libdir];
<<settings.m; (*all the symbols defined here shoudl go in the package context and be not accessible from outside*)
<<lib1.m;   (*same*)
SetDirectory[$rootdir];
);

LoadSettings;

SetDirectory[$moddir];
<<module1.m;
<<module2.m;
SetDirectory[$rootdir];

End[]

EndPackage[]

exampleOLD.nb

$setting1 = 1;
$setting2 = 2;
<< exampleOLD`

As you can see, I would like the nb to set a few variables. Init gets the folder names of tha package to be able to reference absolute paths. Then the main package file should memorise everything in the package context and after that it could even remove those variables from the global one. However after this comes the first problem. If I put the Loadsettings and folder variables in the private part, when loading module1 it will be unable to see those definitions. But the same happens if I put those definitions in the public part. Either way, module1 is unable to load privatelib1.m, or to get the values of any variables in settings.m, lib1.m.

I would also need to understand how to share the function ftoshare and the variable a defined in module2 to module1, and in general in the whole package. This was one of the few thigns that was very good of the new package mode, as from what I understood I only needed to add PackageScope[a] to share a.

$\endgroup$
2
  • 2
    $\begingroup$ I recommend that you localize your actual problems, create the smallest self-contained example to illustrate it (e.g. having just 2 files with very little contents there, just enough to make all your points), and reformulate the question based on that example, providing exact contents of those files in the question as code sections, and also the exact directory structure and exact way you are loading the package. This way, there are much better chances for you to get a useful answer. $\endgroup$ May 19, 2021 at 22:04
  • $\begingroup$ @LeonidShifrin I have edited the question providing more details on how I wanted to struture the whole thing, hope is more clear now. I provided a non-working example using the old packaging mode. all the files in the lib folder would be the files that I have right now, with no modification. The package-format implementation would be done by writing the other files described above. $\endgroup$ May 20, 2021 at 0:31

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.