# Import converter that can import, split, and compress files?

I'm working with a data format that prescribes separate header and data files, so for instance "file.hdr" and "file.dat". They are identified as belonging together by the fact that their base name is the same ("file" in the previous example), and that they reside in the same directories. In practice I will need to import the header before being able to import the data file, but assume for now that they are both simple text files we can easily import. An example of an import converter is given below. The additional Print statement will be used for illustration later.

ImportExportRegisterImport["test", importTest]

importTest[file_, OptionsPattern[]] :=
Module[{base = FileBaseName[file]},

Print[file];

If[StringMatchQ[FileExtension[file], "hdr" | "dat", IgnoreCase -> True],
Check[{Import[base <> ".hdr", "Text"], Import[base <> ".dat", "Text"]}, Return[$Failed]], Return[$Failed]
]

]


Running this on two example files, "file.hdr" (containing the string "hdr") and "file.dat" (containing the string "dat"), gives the expected output:

Import["file.hdr", "test"]

(*Out: "D:\file.hdr"  *)
(*Out: {"hdr", "dat"} *)

Import["file.dat", "test"]

(*Out: "D:\file.dat"  *)
(*Out: {"hdr", "dat"} *)


The problem now lies with the fact that these .hdr and .dat files are often provided in a compressed format, e.g. "file.hdr.gz" and "file.dat.gz". This poses a problem in the example import converter:

Import["file.hdr.gz", "test"]

(*Out: "C:\Documents and Settings\user\Local Settings\Temp\m-xxxx-xxxx_file.hdr" *)
(*Out: $Failed *)  Obviously the "file.dat.gz" file gives similar results. Since Mathematica automatically unpacks compressed files (to the temp directory) before supplying them to their corresponding import functions, as one can see in the example the location of the user-provided file is obscured. My question is thus, how might I be able to import such split compressed files? If somehow I could stop Mathematica from unpacking the compressed files automatically I think the problem would already be solved, but I can't find a way to do this. The only solutions I've come up with now are giving the original file as an option, which will pose an inconvenience for the user, or adding custom downvalues to Import, which gets messy very quickly. - I've resolved the issue, but I can't post an answer yet due to my low reputation score. The solution relies on an undocumented option called "OriginalChannel". I'll post an answer as soon as I can. – tom Aug 9 '13 at 16:11 If I'm not mistaken 1 rep is sufficient to post an answer... – Sjoerd C. de Vries Aug 9 '13 at 22:03 Yes, it was, but I had to wait eight hours from the posting of the question before I could post an answer. – tom Aug 10 '13 at 9:10 ## 1 Answer It took some searching, but I've figured out how solve the issue. It turns out that there's an undocumented option to ImportExportRegisterImport called "OriginalChannel", which if set to True appends the user-given filename or stream to the options supplied to the import converter. An example of a working converter for the described files is thus: ImportExportRegisterImport["test", importTest, "OriginalChannel" -> True] importTest[file_, opts:OptionsPattern[]] := Module[{originalfile = "Channel" /. {opts}}, Which[ StringMatchQ[FileExtension[file], "hdr", IgnoreCase -> True], Check[{ Import[file, "Text"], Import[StringReplace[originalfile, ".hdr" -> ".dat", IgnoreCase -> True], "Text"] }, Return[$Failed]],

StringMatchQ[FileExtension[file], "dat", IgnoreCase -> True],
Check[{
Import[StringReplace[originalfile, ".dat" -> ".hdr", IgnoreCase -> True], "Text"],
Import[file, "Text"]
}, Return[$Failed]], True, Return[$Failed]

]

]


Importing an example file now produces the expected result:

Import["file.hdr.gz", "test"]

(*Out: {"hdr", "dat"} *)


Note that this code is not foolproof, it merely demonstrates a workaround for the problem posted in the question.

-
+1 for sharing your own solution; could you tell us how you found this option? – Mr.Wizard Aug 10 '13 at 9:49
@Mr.Wizard Sure. After being stuck on this for over a day I started to look at some of the Import.m files that come with Mathematica (in SystemFiles\Formats\ in \$InstallationDirectory) and I found that there were options that I didn't recognize ("AlphaChannel" occurred in some image formats for instance). So in a final attempt I started from the top reading all the files, trying to figure out what these undocumented options meant. So finally I came across the Export.m file for the FLV format, and the "OriginalChannel"` option was there, which turned out to do the trick. – tom Aug 10 '13 at 11:43