11
$\begingroup$

In short, I have a package that can load a proprietary device file-format. The underlying reading code is written in Java, packed into .jar, and in a Mathematica package, I can use some simple JLink wrapper functions to access the functionality.

Now, I would like to attach my code to Mathematica's Import, which is usually also trivial by employing ImportExport`RegisterImport. However, to make the importer always available, the RegisterImport code is should be placed in

$UserBaseDirectory/SystemFiles/Formats/MyFormat/Import.m

For simplicity, assume the following fake package structure

MyFormat/
├── Java
│   └── reader.jar
├── Kernel
│   └── init.m
├── Import.m
├── PacletInfo.m
└── ReadingFunctions.m

In ReadingFunctions.m, I'm accessing the Java methods from the jar and provide high-level Mathematica functions for the binary file format. In Import.m I can use these functions and register a custom importer.

My questions are simple:

  1. How can I set-up the PacletInfo.m so that the Import.m goes into the Formats directory and automatically loads the rest of the package when I try to import files?
  2. Or should I use Loading -> "Startup" in the PacletInfo.m to make it available on a kernel start? But then, some unnecessary is always evaluated (e.g. loading of the jar) that I only need if I indeed use the importer in a session.

Are there other/better ways I haven't considered? I could start testing around, but my hope is that someone can give some hints.

Bonus: Do we have a better alternative to this here for adding a file-extension so that it is automatically recognized by Import?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ As for the bonus question: I suspect that there's more to file format detection than the file extension. See FileFormat. Has anyone investigated if overloading FileFormat will affect how Import decides about the format? $\endgroup$
    – Szabolcs
    Commented Jul 12, 2018 at 10:47
  • 1
    $\begingroup$ @GenericAccountName Do you have any hints regarding the main part of the question? $\endgroup$
    – halirutan
    Commented Jul 12, 2018 at 21:51
  • 1
    $\begingroup$ @Szabolcs I don't think overloading FileFormat is really possible for a user without breaking the rest of FileFormat. There currently isn't a user facing way to register a format with FileFormat at all. I will look into whether or not this is something we can add, which would eliminate the need for this.. Sorry for reposting and deleting the comment, there was a typo and apparently you cannot edit comments (actually you can just perhaps only for a certain amount of time?) $\endgroup$
    – GenericAccountName
    Commented Jul 12, 2018 at 22:08
  • 1
    $\begingroup$ @GenericAccountName Comments can be edited for 5 minutes. I think there's nothing wrong with deleting and re-posting if there were no responses to the comment. $\endgroup$
    – Szabolcs
    Commented Jul 13, 2018 at 8:20
  • $\begingroup$ @halirutan did you see my answer? Hopefully this helps, but let me know if you have any questions. I realize the one drawback is after the paclet is installed, you must load the paclet once manually before the import format is registered. After this happens once, Import can just be used like normal. Loading->"Startup" would be a hacky solution to this, I agree always being loaded is not ideal. I'm curious if anyone has an idea so that the $UserBaseDirectory stuff happens when PacletInstall[ ] is called. $\endgroup$
    – GenericAccountName
    Commented Jul 13, 2018 at 18:12

2 Answers 2

Reset to default
4
$\begingroup$

@halirutan you can now register the format extension using this function from Wolfram Function Repository:

https://resources.wolframcloud.com/FunctionRepository/resources/RegisterFormat

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Nice one! That might become handy some day. +1 $\endgroup$
    – halirutan
    Commented Oct 25, 2019 at 22:14
2
$\begingroup$

  1. I don't think there's a way for PacletInfo.m to do this directly, but how about adding something in init.m like this before the Get[]:

    CreateDirectory[FileNameJoin[{$UserBaseDirectory, "SystemFiles", "Formats", "MyFormat"}]];
Export[FileNameJoin[{$UserBaseDirectory, "SystemFiles", "Formats", "MyFormat", "Import.m"}], "RegisterImport...", "Text"];

  2. Rather than registering the paclet functions as a format directly, make a small wrapper that does this, and register myImport instead:

    myImport[filePath_String, opts : OptionsPattern[]] :=
Block[{..},
    Needs["MyFormat`"];
    ...
]

I noted in the comment above that there's no way to hook up to FileFormat (and thus get auto detection of extension/format the proper way) right now so the extension method from this post is the best bet in V11.3

Improve this answer
$\endgroup$
11
  • $\begingroup$ Obviously you could CopyFile an Import.m or some such if you don't want to Export, but that seemed contained :) $\endgroup$
    – GenericAccountName
    Commented Jul 12, 2018 at 22:11
  • $\begingroup$ I think you can get FileFormat to do stuff by registering a few things on the FileFormatDump`$FILEFORMATS and FileFormatDump`$FILEFORMATMATRIX $\endgroup$
    – b3m2a1
    Commented Nov 27, 2018 at 23:55
  • $\begingroup$ @b3m2a1 I highly recommend not modifying these internal variables, it may have unintended consequences. I attempted to come up with a short guide to hack this for power users, but ran into too many issues to make this a feasible approach. $\endgroup$
    – GenericAccountName
    Commented Nov 29, 2018 at 0:44
  • 1
    $\begingroup$ In this case, the source is not available, so I'm not sure how you can do this without some guesses/assumptions. As the maintainer of FileFormat, what I can tell you is the order of FileFormatDump`$FILEFORMATS matters and we've decided it's not a good idea to modify these for now. I would suggest writing a wrapper that calls Import with the correct format if the extension or some other criteria about the file is met (magic number, heuristics, etc), and passes the data untouched to Import otherwise. Or even Unprotect Import, like the example I posted in my answer shows. $\endgroup$
    – GenericAccountName
    Commented Nov 29, 2018 at 18:34
  • 1
    $\begingroup$ If you absolutely insist on such a hack in this direction, I suggest at least adding new formats to position -2 in $FILEFORMATS. Obviously with the disclaimer that this is not documented, thus unsupported and unexpected behavior may occur. If you haven't figured out how FileFormatDump`$FILEFORMATMATRIX works, first get FileFormatDump`$FILEFORMATMATRIX[FileFormatDump`$FILEFORMATS[[-2]]], and set FileFormatDump`$FILEFORMATMATRIX["YourFormat"] with slight modifications from the example, ignoring the booleans. $\endgroup$
    – GenericAccountName
    Commented Dec 2, 2018 at 2:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.