# EndOfFile error

I have a LARGE file (14.6 GB) that I want to separate into several smaller files. Because of its size, using Import/Export is not feasible, I think the most reasonable way of handling this would be using the file as a stream

stream = OpenRead[filepath]


then with

Read[stream, String]


I can select what I want, and hold the strings in a temporary list until that is large enough to warrant being exported and cleared.

The problem is that when Read[stream,String] encounters ASCII character 0x1A, Mathematica interprets it as an EndOfFile, even when it's not. Any suggestions to get around this?

Edit: https://www.dropbox.com/s/sm51m2s19wtnoqa/allascii.txt is a text file with all ASCII characters, if read in as a stream, this error should be reproduced

-
Looks like you need BinaryReadList. –  Sjoerd C. de Vries Jun 28 '14 at 6:56
1A is really a kind of EOF: see this. –  Sjoerd C. de Vries Jun 28 '14 at 7:02
Is this a binary file or a plain text file? Plain text files shouldn't have that character ... if it's a binary file, use BinaryFormat -> True in OpenRead. –  Szabolcs Jun 28 '14 at 13:54

Long, long ago, but right here and not in a galaxy far, far away, there was an operating system called MS-DOS that used the character 0x1A as an end-of-file flag. Windows inherited this in its early days. I would have thought that it was eliminated by now, but perhaps not. Mathematica is old enough that it has a system constant $IgnoreEOF. It should be True by default, but it isn't, so $IgnoreEOF = True