# Read a binary file to an image in memory then analyze it

Is it possible to read a whole file (~100 MBs) into memory, and then use BinaryRead[] function on the data loaded into memory to read/cast it to the types I need with a proper StreamPosition pointer?

I need this because the process is very slow if I do it directly to files, so I prefer to have all manipulations done in memory, especially that I require many memory accesses with many BinaryRead[] calls.

Thank you for any efforts.

-
have you looked into setting up a memory resident virtual file system ? system dependent obviously... –  george2079 Jan 21 at 13:28
@bills it's not only integers. There are various types I wanna read, and I can't find any casting functionality in Mathematica, that's why I'm asking. –  The Quantum Physicist Jan 22 at 10:18

take a look at this:

Create a binary file with mixed types:

 f=OpenWrite["tmp2",BinaryFormat->True];
BinaryWrite[f,4545.4343,"Real64"];
BinaryWrite[f,"abc","TerminatedString"];
BinaryWrite[f,354245,"UnsignedInteger32"];
Close[f];


read it back in as all "Byte"

f=OpenRead["tmp2",BinaryFormat->True];
Close[f];


recover the data:

BinaryRead[ StringToStream[StringJoin@(FromCharacterCode[#]&/@
bytes[[1;;8]])]  , "Real64" ]
bytes[[9;;12]])]  , "TerminatedString" ]
bytes[[13;;16]])]  , "UnsignedInteger32" ]

->  4545.4343 , "abc" , 354245


I don't know why you'd do this vs just reading in the proper format in the first place, unless of course you don't know the format in advance.

Edit..just noticed this can be expressed a bit more compactly, eg.

 BinaryRead[StringToStream@FromCharacterCode@bytes[[13;;16]]  ,
"UnsignedInteger32" ]


Caveat --- Each "Byte" is stored by mathematica as a 64 bit integer, so your memory usage is 8x the file size.. Don't know a clean way around that..

Edit 2 here's the fix.. stuff everything into a big string:

buf=FromCharacterCode@BinaryReadList[f,"Byte"];


Then read back like this:

BinaryRead[StringToStream@StringTake[buf,{13,16}],"UnsignedInteger32"]

-