I want to read in large files as byte strings.
Import[..., "Byte"]
yields a list of Integers that takes up much more space (16 times more) than the file.
ByteArray
would be the proper way to store such sequences in memory, but how can I read a file into a ByteArray
directly?
I know I can later apply ByteArray
to the result of Import
, but I would prefer to not use so much memory intermittently.
Importing as Integer64
uses much less extra memory, but it's not an option because it does not read in everything when the size is not divisible by 4.
Importing as "String"
seems to not use much more memory than the file, but I am not sure whether this might lead to problems since I have arbitrary byte sequences. I'm also not sure how I would index the n-th byte (rather than the n-th multibyte character).
String does store multibyte variable length characters (utf-8 style), right? It's not just a wchar_t/short/int array, is it?
Any ideas?
"String"
, only 8-bit "characters" are returned (i.e. characters represent bytes) $\endgroup$