Timeline for Declaring Package with dependencies in multiples files?
Current License: CC BY-SA 4.0
26 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 22, 2018 at 7:46 | history | edited | Szabolcs | CC BY-SA 4.0 |
added 3 characters in body
|
| Jul 30, 2018 at 5:59 | comment | added | Leonid Shifrin |
@RonaldMonson Re: autoloading: if you mean new-style packages, one can still do the autoloading. One way is via paclets, as you mentioned. But I would have to look closer to say more, in particular it might indeed be that DeclarePackage would not work. Re: Loading any file in the package directory seems to trigger a loading of all the files - good point. Yes, this is true, and therefore, one should not attempt to load all fragments via Get in the init.m for new-style packages. That would not lead to an infinite loop, but each file would be loaded N times. Load just one file instead.
|
|
| Jul 30, 2018 at 5:34 | comment | added | Ronald Monson | @LeonidShifrin Is there a gotcha regarding ordering loading via Kernel/init.m? Loading any file in the package directory seems to trigger a loading of all the files in the directory as well as the Kernel/init.m file thereby creating an infinite loop. | |
| Jul 30, 2018 at 5:29 | comment | added | Ronald Monson |
@LeonidShifrin Ok, thanks; +1 for bringing this to the community's attention and clarifying comments. One thing about the automatic loading though; it occurs to me that despite the convenience gained, what you are losing is potentially v. significant - the ability to load on demand via DeclarePackage (there is, I see the "AutoloadSymbols" key supplied to PacletManager`Package`loadWolframLanguageCode which possibly addresses this when PacletInfo.m is present (?) but loading on demand still doesn't seem infused).
|
|
| Jul 18, 2018 at 16:06 | comment | added | Leonid Shifrin | @RonaldMonson I agree with most of your points. As to new package format, it is undocumented, so I don't think there is any walk through. But my description should be enough to start, and also there are components in Mathematica distribution present in their source form (rather than MX), written in this style - which is where one can also look for how it is used. | |
| Jul 18, 2018 at 10:05 | comment | added | Ronald Monson | @LeonidShifrin [3/3] Finally, it's a bit tantalizing to equate npf with an experimental package that has been around for a few years when master spelunkers like b3m2a1 and Szabolcs continue to spelunk :) Does there perhaps exist a simple walk through (packages to load etc.) for users wanting to give it a try? | |
| Jul 18, 2018 at 10:04 | comment | added | Ronald Monson |
@LeonidShifrin [2/3] ... (clarity of thought, smaller, non-corrupting files, versioning etc) but doesn't enforce "excessive" "PackageScope" exports until users are ready to "fragment by context". Conversely, being able to transit from "fragmenting by (private) context " to "fragmenting by fully-fledged package" perhaps also encapsulates a next fragmenting stage that in turn might contribute to attaining "true, multi-level modularization" needed for npf's scaling.
|
|
| Jul 18, 2018 at 10:03 | comment | added | Ronald Monson |
@LeonidShifrin [1/3] Yes well "splitting into subpackages only because you want to split into fragments" pretty much sums up a lot of my workflow so IMP therein lies the real value of this new package format (npf): The potential to fragment with finer granularity that might also lead to transitions from functions->fragments->(sub)packages. Pursuing this, and for those wanting to run the name-colliding gauntlet, it would be great to have a token in a fragment that specifies the same MyPackage`Private` context. This would permit an earlier, "fragmentation by file" that has its own value ...
|
|
| Jul 15, 2018 at 19:41 | comment | added | Leonid Shifrin |
@RonaldMonson When you split a project into several fragments, it becomes very easy to make collisions for private symbols in different fragments, when the code grows large. Single Private` subcontext is then hardly sufficient. Splitting into subpackages only because you want to split into fragments doesn't sound like the best solution generally, and the new-style package allows one to do the latter without the former. Needs still works the same way. For DeclarePackage I would need to check, but I think it should work the same way too, for new-style packages.
|
|
| Jul 15, 2018 at 18:36 | comment | added | Ronald Monson |
@LeonidShifrin [2/2] ... for me, name collisions within Private contexts has never really been an issue since well before they might become one the code has been naturally fragmented into (sub)Package(s); This also raises the need for a Find/Replace mechanism across files in the same directory and I also wonder about losing Need's ability to avoid double-loading and where DeclarePackage fits in?
|
|
| Jul 15, 2018 at 18:34 | comment | added | Ronald Monson |
@LeonidShifrin [1/2] doesn't Private already provide this encapsulation so that the better modularization is "only" to the extent that BeginPackage/Begin wrappers and Need's calls (for the same-directory fragments) are being dispensed with? I agree that their remains much upside still available from greater modularization in general Mma code so perhaps this simplification should not be underestimated (not to mention advantages from having available softer, developer-focussed exporting). On the other hand ...
|
|
| Jul 15, 2018 at 12:47 | comment | added | Leonid Shifrin | @RonaldMonson There are two points which I outlined there: better encapsulation for private symbols in multi-fragment files and somewhat better modularization, achieved by the package-scope level and complete insulation of private contexts of different fragments. These things together lead to better code, and are IMO a step in the right direction. How huge that step is, depends on who you ask, but I personally like the new format much better exactly for above reasons. | |
| Jul 15, 2018 at 9:35 | comment | added | Ronald Monson | All the "problems solved" by this new approach seem to be problems inherent in the "Master-Slave" approach but not in the standard "Namespace-BeginPackage" approach while all the "problems still not solved" are ones that seem to be already solved in the "Namespace-BeginPackage" approach - hence I can't quite see why you consider this to be a "huge step in the right direction"? | |
| Jul 4, 2018 at 17:23 | comment | added | Szabolcs | Here's a related question: mathematica.stackexchange.com/questions/176627/… | |
| Jul 3, 2018 at 17:09 | comment | added | Leonid Shifrin | @SumNeuron I will try to expand my answer in that direction in the next day or two. | |
| Jul 3, 2018 at 17:08 | comment | added | Leonid Shifrin | @Kuba There is currently an effort underway internally to solve these problems and come up with a modern package format. I however don't think that we have to wait for the moment when that new format emerges, gets tested and becomes official - that can easily take another year or two. We can start by creating a chat room here, devoted to this topic, or may be create a Slack channel (which is probably a better option). I will try to expand my answer in the next few days, to give more details on my current way of doing multi-package projects. | |
| Jul 3, 2018 at 17:01 | comment | added | Leonid Shifrin |
@Kuba Probably true. But I will need to think it over to have a more definite opinion. A good solution should implement proper versioning, deal with autoloading, also some dependencies may be MX-fied (in which case they will load directly into certain contexts, which probably means that the versioning scheme should also be wired into the distribution mechanism (basically, loader is not enough, one would also need MX-fier that would put things in proper contexts like GeneralUtilities$10$1$3` ( I used $ sign as a version index separators). There are other things to consider too.
|
|
| Jul 3, 2018 at 15:32 | comment | added | SumNeuron | @LeonidShifrin I appreciate the detailed answer (still going through it and trying to understand it) and would also be interested in the mutli-package... because I still do not see how to do relative imports of modules / packages | |
| Jul 3, 2018 at 14:09 | comment | added | Kuba |
@LeonidShifrin Custom loader could overload BeginPackage and friends to load as TemplateApply["GeneralUtilities<*RandomInteger[10^15]*>`"], then having that we need to agree to a postfix convention for sem ver paclets and we are done. Few assumptions about package coding style would be needed but otherwise seems plausible to do, what do you think?
|
|
| Jul 2, 2018 at 20:04 | comment | added | Leonid Shifrin |
@Szabolcs [2/2] ... the main project. They are allowed to throw exceptions. All proxied public functions catch all inner exceptions. So, in your example, we would have public interface, plus 3 sub-modules A, B and C. Both A and B will have public function which will depend on C (its exported public functions). If we want to expose some of the public functions from A and / or B, we create proxies for them (on a very basic level, a proxy is just a wrapper like PublicF[args___]:= A`F[args], and expose that. We have a constructor for the proxy, which is something like makeProxy[def].
|
|
| Jul 2, 2018 at 20:00 | comment | added | Leonid Shifrin | @Szabolcs [1/2] What we do in my current project is to split the project into several modules, each of which is an independent package with context being a sub-context of the main one. Then we have public interface of the project. Modules may depend on other modules, that's perfectly fine. Whenever I need to expose some of the internal module's function in some way as a public function, I define a proxy wrapper, that more or less delegates to that module's function, adding some error-handling. This process can be automated. All public functions for modules are considered inner functions for... | |
| Jul 2, 2018 at 19:54 | comment | added | Leonid Shifrin | @Kuba Can't help with versioned paclets, they don't exist yet AFAIK. Including third-party packages as internal ones - won't work either at the moment, if you want generality, because currently there is no way to have two incompatible versions of the same package loaded into the system at the same time without conflicts. But point taken, I will try to expand at least with a few more things that might be useful. | |
| Jul 2, 2018 at 19:40 | comment | added | Szabolcs | Suppose that the functions of a package naturally split into two sets. Let's take an attempt at modularization and break this package into two packages. But then I discover that both sets of public functions depend on a common set of private functions that will never be exposed. What's the best way to proceed? | |
| Jul 2, 2018 at 18:32 | comment | added | Kuba | Don't know how much interest is enough so let me say that I am interested. Especially if that would cover versioned paclets or including 3rd party packages as internal ones (unique context etc). | |
| Jul 2, 2018 at 18:18 | history | edited | Leonid Shifrin | CC BY-SA 4.0 |
Added a few more details, structure
|
| Jul 2, 2018 at 16:52 | history | answered | Leonid Shifrin | CC BY-SA 4.0 |