3
$\begingroup$

I'm trying to open several large .txt files which are located in a GZIP file. I am able to import each separate file using Import, but this is too slow. I do it like this:

dataFileNames = Import["file.gz","FileNames"];
dataImport[i_] := 
  Import["file.gz",{dataFileNames[[i]],"Table"]},"FieldSeparators"->";"]

Is there a way to apply the ReadList (or BinaryReadList) to make importing the files go faster? I prefer to not have to extract the files as this would take up a lot of space.

$\endgroup$

closed as off-topic by Szabolcs, Henrik Schumacher, MarcoB, eyorble, Yves Klett Sep 24 '18 at 13:27

  • The question does not concern the technical computing software Mathematica by Wolfram Research. Please see the help center to find out about the topics that can be asked here.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Do you have a zip or a gzip? They're not the same thing. gzip only compresses a single file, it makes no sense to get the "FileNames", and it doesn't even work when I try it. Thus before answering I need to make sure that you have gzip files. Normally, the original extension is also kept when gzip'ping, e.g. file.csv.gz. Import will transparently decompress and treat such a file as a CSV. $\endgroup$ – Szabolcs Sep 17 '18 at 7:34
  • $\begingroup$ BTW ReadList can be made to transparently decompress a gzip. You need to OpenRead the file with the appropriate input stream Method (see $InputStreamMethods), then read from the stream, then Close it. But I don't want to write an answer until you clarify the question. The code you cite doesn't work for me at all. $\endgroup$ – Szabolcs Sep 17 '18 at 17:33
  • 4
    $\begingroup$ I'm voting to close this question as off-topic because it is unclear and OP is unresponsive to clarification requests. $\endgroup$ – Szabolcs Sep 18 '18 at 13:37
0
$\begingroup$

I have a similar answer on Importing a file inside Zip with OpenRead

First off, gzip is not an archive format, it is a compression format, so "FileNames" will only ever return the one file.

Try:

strm = StringToStream[Import["file.gz", "String"]]

Then you can use ReadList etc. Like @Szabolcs mentions, Import will transparently decompress.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.