Import converter that can import, split, and compress files?

Question

I'm working with a data format that prescribes separate header and data files, so for instance "file.hdr" and "file.dat". They are identified as belonging together by the fact that their base name is the same ("file" in the previous example), and that they reside in the same directories. In practice I will need to import the header before being able to import the data file, but assume for now that they are both simple text files we can easily import. An example of an import converter is given below. The additional Print statement will be used for illustration later.

ImportExport`RegisterImport["test", importTest]

importTest[file_, OptionsPattern[]] := 
 Module[{base = FileBaseName[file]},

  Print[file];

  If[StringMatchQ[FileExtension[file], "hdr" | "dat", IgnoreCase -> True],
   Check[{Import[base <> ".hdr", "Text"], Import[base <> ".dat", "Text"]}, Return[$Failed]],
       Return[$Failed]
  ]

 ]

Running this on two example files, "file.hdr" (containing the string "hdr") and "file.dat" (containing the string "dat"), gives the expected output:

Import["file.hdr", "test"]

(*Out: "D:\file.hdr"  *)
(*Out: {"hdr", "dat"} *)

Import["file.dat", "test"]

(*Out: "D:\file.dat"  *)
(*Out: {"hdr", "dat"} *)

The problem now lies with the fact that these .hdr and .dat files are often provided in a compressed format, e.g. "file.hdr.gz" and "file.dat.gz". This poses a problem in the example import converter:

Import["file.hdr.gz", "test"]

(*Out: "C:\Documents and Settings\user\Local Settings\Temp\m-xxxx-xxxx_file.hdr" *)
(*Import::nffil: File not found during Import.                                   *)
(*Import::nffil: File not found during Import.                                   *)
(*Out: $Failed                                                                   *)

Obviously the "file.dat.gz" file gives similar results.

Since Mathematica automatically unpacks compressed files (to the temp directory) before supplying them to their corresponding import functions, as one can see in the example the location of the user-provided file is obscured. My question is thus, how might I be able to import such split compressed files?

If somehow I could stop Mathematica from unpacking the compressed files automatically I think the problem would already be solved, but I can't find a way to do this. The only solutions I've come up with now are giving the original file as an option, which will pose an inconvenience for the user, or adding custom downvalues to Import, which gets messy very quickly.

Answer 1

It took some searching, but I've figured out how solve the issue. It turns out that there's an undocumented option to ImportExport`RegisterImport called "OriginalChannel", which if set to True appends the user-given filename or stream to the options supplied to the import converter. An example of a working converter for the described files is thus:

ImportExport`RegisterImport["test", importTest, "OriginalChannel" -> True]

importTest[file_, opts:OptionsPattern[]] := 
 Module[{originalfile = "Channel" /. {opts}},

  Which[

   StringMatchQ[FileExtension[file], "hdr", IgnoreCase -> True],
   Check[{
     Import[file, "Text"], 
     Import[StringReplace[originalfile, ".hdr" -> ".dat", IgnoreCase -> True], "Text"]
   }, Return[$Failed]],

   StringMatchQ[FileExtension[file], "dat", IgnoreCase -> True],
   Check[{
     Import[StringReplace[originalfile, ".dat" -> ".hdr", IgnoreCase -> True], "Text"], 
     Import[file, "Text"]
   }, Return[$Failed]],

   True,
   Return[$Failed]

  ]

 ]

Importing an example file now produces the expected result:

Import["file.hdr.gz", "test"]

(*Out: {"hdr", "dat"} *)

Note that this code is not foolproof, it merely demonstrates a workaround for the problem posted in the question.