13
$\begingroup$

What I know:

  • PackedArray is documented (to some extent) in "Developer Utilities Package", RawArray is just mentioned in Raster3D and ImageApply,

  • RawArray supports Byte, SignedInteger8/16/32, Bit16, Real32, Real and Complex128 as entries, while PackedArray only Integer, Real and Complex(128),

  • PackedArray displays as a list but RawArray as (somewhat of) a skeleton.

What I do not understand is why there are two of them. Why not just stick with the one that supports more entry types and display it as skeleton if and only if it is too long?

Also, how does ByteArray fit into the picture?

Edit: You can create a PackedArray using:

ap = Developer`ToPackedArray[{1, 2, 3}]
(* {1,2,3} *)
Developer`PackedArrayQ[ap]
(* True *)
Developer`PackedArrayForm[ap]
(* PackedArray[Integer, <3>] *)

(Note that there is in fact no PackedArray command, the head of the last output expression is a string.)

You can create a RawArray using:

ar = Developer`AllocateRawArray["Byte", {2, 2}]
(* RawArray[Byte, <2,2>] *)
Developer`RawArrayQ[ar]
(* True *)
Developer`RawArrayType[ar]
(* Byte *)

(ditto about RawArray).

You can create a ByteArray directly:

ab = ByteArray[{1,2,3}]
(* ByteArray[<3>] *)
ByteArrayQ[ab]
(* True *)

All the cross-tests (e.g., RawArrayQ done on ab) return False.

$\endgroup$
  • $\begingroup$ no idea what this means, but according to ByteCount both packed and raw arrays use at least 8 bytes per value, even with the "Byte" type explicitly called out. ByteArray uses the expected one byte per value. The raw array seems to be always exactly 96 bytes bigger than the packed array. $\endgroup$ – george2079 Mar 16 '16 at 19:02
  • $\begingroup$ PackedArray can't use bytes, 8 bytes is the word size on a 64-bit machine so that makes sense. RawArray works well for me: ar = Developer`AllocateRawArray["Byte", {10000}]; ByteCount[ar] gives just a little over 10000. It's 176 rather than 96 bytes of overhead on my computer. $\endgroup$ – The Vee Mar 16 '16 at 19:04
  • $\begingroup$ I see. I was doing AllocateRawArray["Byte", *biglist* ] which is invalid so just getting the size of the list. $\endgroup$ – george2079 Mar 16 '16 at 19:20
  • $\begingroup$ @george2079 instead of *biglist* you should use a dimension specification, e.g. {40,50} is a 40 by 50 matrix. $\endgroup$ – Szabolcs Mar 17 '16 at 11:36
  • $\begingroup$ yes i figured that out. I should add on my machine ByteCount and MemoryInUse indicate hardly any memory usage after "allocating" a large array. $\endgroup$ – george2079 Mar 17 '16 at 12:27
15
$\begingroup$

Here goes my conjecture but I would welcome any opinion or clarification by people in the know.

  1. PackedArray stores data in a computation-efficient way. It still represents a List in all other respects, and is displayed as one. Some Lists are generated as PackedArrays silently where Mathematica decides it possible and helpful, and unpacked if an operation incompatible with the packing is performed (like changing one element for another of a different type, out of the range of the type, or for a symbolic expression). The reason to store Integers, Reals and Complexes only is that these are natively supported for computation. The actual size will also depend on the system implementation (32- or 64-bit). You can perform mathematical operations on a PackedArray and it will stay packed and will presumably vectorise the operations where possible.

  2. RawArray stores data in a memory-efficient way in a wide variety of formats that may be useful for different purposes. (Storing a Byte instead of a small Integer indeed saves a lot of memory (88% on a 64-bit system) but one would need additional CPU instructions in order to calculate with bytes.) It does not represent a list as it is considered a zero-depth object: ar[[1]] will give an error (and then, unwillingly, tell you it's actually "Byte" in the OP example). ar[[2]] gives yet another special object called an ArrayObject which indeed has no structure as far as Mathematica is concerned. It follows from this that one can't extract data from the RawArray using Part, and can not perform mathematical operation with these. RawArray is ideal for (and is used for) storing special-purpose data in objects representing big amounts of very regularly structured information like bitmap images and sound samples. Only functions actually designed to deal with the specific kind of data access and modify the information stored inside, otherwise the block of data it represents is constant.

  3. ByteArray is roughly half the way between these: it stores data in a memory-efficient way, with no direct computation, but still representing a list of data which can be accessed directly using Part. As opposed to the other two it is limited to a linear (one-dimensional) lists and thus perfect for compressed streams and the like. It can also be converted to a normal list using Normal, like other optimized data structures. It turns out that a ByteArray is actually identical to an ArrayObject[Byte, _], but this is not manifested externally. Also, the latter supports a wider selection of types than just Bytes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.