Are you on a Windows machine? If so, try
OpenReading the file with
When you open a file on a Windows machine in "text mode," with
BinaryFormat->False (which is the default), the system has to translate the Windows newline sequence
\r\n (two bytes which mark the end of the line) into the single
\n character. In the C language and in Mathematica,
\n (one byte) marks the end of a line.
As a result, there is a translation step that must happen. Even if you use
SetStreamPosition[stream, n] it must work out where character n is, which requires looking at all the intervening characters. For gigantic files, this can take some time. (It builds a table in memory so it only has to work it out once.)
If you open the file with
BinaryFormat->True, you will be responsible for handling the difference between
\r\n yourself. But in that case, there is no translation, and
SetStreamPosition is essentially instantaneous. Since you know the exact length of each line (adding 1 extra byte for each newline on Unix, 2 extra bytes on Windows!) you can compute exactly where
SetStreamPosition should go.
Another way, which doesn't require
BinaryFormat->True, is to "pre-load" the translation step. After you open the file for the first time, do
SetStreamPosition[stream, Infinity] to set the stream position to the end of the file. In the process, all the newlines will be scanned along the way. That's slow at the beginning, but after that point any calls to
SetStreamPosition will be fast, because it will have already built the table.
Skip when you have fixed size lines -- that's a waste.
Skip is really just
Read without saving the data that's read. It's appropriate when you don't know how long each record will be and you actually have to process the record to find the next one.
If you're not on a Windows machine I have no idea why
SetStreamPosition would be slow.