How can I create function, that will not unpack packed arrays with Map/Table. For example I want read bytes from opened stream (this is my original goal):
getBytesFromStream[stream_, pos_, len_, converter_] :=
Module[{bytes},
SetStreamPosition[stream, pos];
bytes = BinaryReadList[stream, "Byte", len];
converter[bytes]
]
bytes =
Module[{stream = OpenRead["test.dat", BinaryFormat -> True], bytes},
bytes = getBytesFromStream[stream, 0, 128, #&];
Close[stream];
bytes
];
bytesList =
Module[{stream = OpenRead["test.dat", BinaryFormat -> True], bytes},
bytes = Table[getBytesFromStream[stream, i, 128, #&], {i, 0, 127, 128}];
Close[stream];
bytes
];
Developer`PackedArrayQ[bytes]
Developer`PackedArrayQ[bytesList]
True
False
Table did unpack array that was be read. I can pack it again. But if I have only 1Gb free memory and if I want to read file that has size about 400~500Mb - Mathematica can not process this. Unpacked 512Mb in the Mathematica memory has size about 1536Mb. Therefore, I want the function to return the packed result.
Table[ConstantArray[0., 3], {10}] // Developer`PackedArrayQ
andIdentity /@ ConstantArray[0., {10, 3}] // Developer`PackedArrayQ
. This stream business is actually not related to the question. Agreed, this is a very annoying issue. $\endgroup$ConstantArray
) and writing into it should resolve the issue. If not, loading smaller chucks into preallocated arrays asJoin
ing them afterwards should also work fine. $\endgroup$