Mathematica Stack Exchange is a question and answer site for users of Mathematica. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is there a simple way to use Integer types (in particular, a table of Integer types) that require only 4 or 8 bytes (reps. 32 and 64 bits) representation?

It seems that integers in Mathematica have a 192 bits representation:

ByteCount[RandomInteger[]]
24

Using a 64 bits representation would save up lots of memory in problems where you know you don't require too large integers and memory is an issue.

Note that this question applies to other types.

share|improve this question
    
fyi, I get 16 on windows 7, 64 bit, version 9.01 !Mathematica graphics – Nasser Jun 27 '14 at 0:39
    
really !?, I use Mathematica 8.0.4.0 on MacOSX x86 – VanillaSpinIce Jun 27 '14 at 0:45
4  
You might want to look at packed arrays: mathematica.stackexchange.com/questions/3496/… – acl Jun 27 '14 at 1:24
    
I also get 16 on running V9.0.1 on OS X – m_goldberg Jun 27 '14 at 6:06
    
I get 16 on V9.0.1 on Windows 8.1 (x64, of course). – m0nhawk Jun 27 '14 at 6:15

In general, there's little control over this, but here are the options:

For efficient storage of numerical data, Mathematica uses packed arrays. Within a packed array, an integer takes up 8 bytes (64 bits) when using a 64-bit version of Mathematica and 4 bytes when using a 32-bit version. This is documented in the LibraryLink reference (mint type). With some care you can make sure that all your arrays are kept in packed form.

Image objects are atomic since (I think) version 9 and support several storage methods, including bit, byte, 16-bit integer, 32-bit real and 64-bit real (ImageType). Once again these are directly accessible from LibraryLink, so the LibraryLink reference details the internal storage formats.

Since version 10.1 we also have the ByteArray atomic type for efficient storage of bytes.

I believe that Image and ByteArray are not meant for general numerical data, but they might fit your specific application.


What you are measuring in your example is not a packed array, but a general Mathematica expression. Due to their generality, these need (much) more storage. A list can contain not only integers, but any other sort of expression, thus each expression must individually encapsulate some type information.

The internal storage format is not documented and we can only speculate about it. So let me speculate a bit: if I wanted to implement expressions, for an integer list I would need in the least:

  1. Store the integer itself. For a machine size integer 8 bytes are a reasonable choice on 64-bit platforms.
  2. Support large integers that don't fit into 64 bits, or other types. This means that I would need to attach some flag to the integer to indicate its type and storage format. If I use a machine integer for this flag, it adds an extra 8 bytes.
  3. For each element of a List (or any other head), we need to store a pointer that can point to any sort of Mathematica expression. On a 64-bit platform this adds 8 more bytes.

Thus I might end up having to use up an extra 24 bytes for each new machine integer added to a list. This is indeed what seems to happen for small lists: ByteCount[{1,2,3}] - ByteCount[{1,2}] is 24. To further support the idea, on my machine ByteCount[1] (i.e. an integer not in a list) gives 16.

All of this is of course only speculation and it's easy to find evidence that in practice the implementation is more complicated. Try plotting e.g.

 Table[
  ByteCount@Developer`FromPackedArray@ConstantArray[2, n + 1] - 
  ByteCount@Developer`FromPackedArray@ConstantArray[2, n],
  {n, 1, 100}]

Mathematica graphics

The mean of these values is about 24, but the individual ones may be much larger. My naive theory to explain that is that sometimes Mathematica allocates more memory than needed to allow the list to grow efficiently, if necessary. Or it might be something else.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.