3
$\begingroup$

I'm trying to open several large .txt files which are located in a GZIP file. I am able to import each separate file using Import, but this is too slow. I do it like this:

dataFileNames = Import["file.gz","FileNames"];
dataImport[i_] := 
  Import["file.gz",{dataFileNames[[i]],"Table"]},"FieldSeparators"->";"]

Is there a way to apply the ReadList (or BinaryReadList) to make importing the files go faster? I prefer to not have to extract the files as this would take up a lot of space.

$\endgroup$
  • $\begingroup$ Do you have a zip or a gzip? They're not the same thing. gzip only compresses a single file, it makes no sense to get the "FileNames", and it doesn't even work when I try it. Thus before answering I need to make sure that you have gzip files. Normally, the original extension is also kept when gzip'ping, e.g. file.csv.gz. Import will transparently decompress and treat such a file as a CSV. $\endgroup$ – Szabolcs Sep 17 '18 at 7:34
  • $\begingroup$ BTW ReadList can be made to transparently decompress a gzip. You need to OpenRead the file with the appropriate input stream Method (see $InputStreamMethods), then read from the stream, then Close it. But I don't want to write an answer until you clarify the question. The code you cite doesn't work for me at all. $\endgroup$ – Szabolcs Sep 17 '18 at 17:33
  • 4
    $\begingroup$ I'm voting to close this question as off-topic because it is unclear and OP is unresponsive to clarification requests. $\endgroup$ – Szabolcs Sep 18 '18 at 13:37
0
$\begingroup$

I have a similar answer on Importing a file inside Zip with OpenRead

First off, gzip is not an archive format, it is a compression format, so "FileNames" will only ever return the one file.

Try:

strm = StringToStream[Import["file.gz", "String"]]

Then you can use ReadList etc. Like @Szabolcs mentions, Import will transparently decompress.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.