# Read and filter a text file without importing

I have a large text file of a million sections. Each section contains A lines of header and B lines of numeric data. I want to read the first and last line of B from each section (to have a list of 2 x million lines at the end). Import doesn't work since my file is too big to be read into memory. Is there a command like scan and filter for this?

• You may try for example SetStreamPosition[], but ... why don't you just preprocess the file with any capable language for text processing (AWK, perl, sed, whatever) Feb 11 '13 at 14:40
• Have a look at this question. There are a number of solutions shown there, which you might be able to adopt to your situation. All of them will however require to write some custom code. Feb 11 '13 at 15:05
• Possibly a duplicate of this question. The link is certainly relevant to the current question. Feb 11 '13 at 16:58
• Was my answer useful to you? Feb 22 '13 at 23:16

You're looking for Skip. This does not check for an EndOfFile condition, and should use something like BlockStream to handle aborts, but the following should work:

strm = OpenRead["filename"];

(* Repeat the following until done *)

Skip[strm, String, A];