I have a LARGE file (14.6 GB) that I want to separate into several smaller files. Because of its size, using Import/Export is not feasible, I think the most reasonable way of handling this would be using the file as a stream
stream = OpenRead[filepath]
I can select what I want, and hold the strings in a temporary list until that is large enough to warrant being exported and cleared.
The problem is that when Read[stream,String] encounters ASCII character 0x1A, Mathematica interprets it as an EndOfFile, even when it's not. Any suggestions to get around this?
Edit: https://www.dropbox.com/s/sm51m2s19wtnoqa/allascii.txt is a text file with all ASCII characters, if read in as a stream, this error should be reproduced