Import selected columns from a tab delimited text file

Question

I have a very huge file (1GB) from which I desire to import only the 5th column. It is a tab delimited text file with a .txt extension. Can you please help me with this?

I generally use Import["filename.txt","Table"] to import my files, but now, I don't want the whole file, which takes a long time to load.

Answer 1

On Linux you could use this simple solution.

Linux Shell

From the shell:

$cat myfile.txt | cut -f 5 > col5.txt

Then import into Mathematica as usual.

data = Import["col5.txt","Table"]

ReadList

This can be condensed to one ReadList command.

data = ReadList["!cat myfile.txt | cut -f 5"];

Answer 2

You might try this:

Import["data.txt", {"Data", All, 5}]

If you want to load only the fifth column of the first two lines you can run this:

Import["data.txt", {"Data", {1, 2}, 5}]

You can find more about this here.

Update

Some testing on a relatively small data file (12MB) might prove that this is not really the solution you're after. It seems that it takes much more time to load just one column (~130 seconds) than to load the entire file with Import["data.txt","Table"] (~4 seconds).