I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows.
So for small data files I have just been using:
dataTable = Import["data.txt", "Table"];
However once the files gets so large (into the millions), Mathematica and my computer slow down considerably. Now, I don't really care about most of the data in these files and in Mathematica I end up only using several thousand entries. So my question is, can I import a random sampling from the large data file (say 10000 elements only) instead of importing the entire file? So if I had a file with 10^6 rows, I would like to import just 10^4 of those rows (preferably randomly).