# Is it possible to lower the bitwidth of (floating point) numbers?

The answer to this question will most probably be a definite no. Nevertheless, I would like to know whether it is possible to represent reals in the memory on less than the standard 32 or 64 bits (depending on machine). This limits my exploration to $\sim 2^{14} \times2^{14}$ matrices, though with half or quarter precision I could venture a bit further. The matrix itself cannot be broken down to smaller pieces for further compuation, and it cannot be reworked to a simpler one, but high precision is of no importance. The question is not about refactoring matrices but about number representation and memory handling by Mathematica.

Warning, the following computation may eat through your memory limits.

n = 14;
all = (2^n)*(2^n);
m = RandomReal[{0, 1}, {2^n, 2^n}];
{all, (ByteCount@m/1024./1024) "MB", ((ByteCount@m/all*8.) // N) "bit"}
ClearAll[m];

{268435456, 1024. "MB", 64. "bit"}


I am using 64-bit Win7, with 2 GB RAM, and (expectedly) my system almost chokes on this code. And then of course I hadn't calculated anything with the matrix yet...

-
High precision may be of more importance than it appears, since for large matrices, at least in some cases, one may get large accumulated errors in the computations - this of course depends on the matrix and on the operations you want to do with it. –  Leonid Shifrin Apr 4 '13 at 9:47
@Leonid Yes, I am aware of that, but still I wonder if it is possible to represent my data on less then the standard amount of memory. Or should I switch to C (or similar) if I want half precision... –  István Zachar Apr 4 '13 at 9:49
You could give SetPrecision a try. But for a single float, this increases the memory usage from 24byte to 72byte. –  Stefan Apr 4 '13 at 9:51
SetPrecision[...,something small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking. –  Daniel Lichtblau Apr 4 '13 at 13:36
Try the following using out-of-core memory. (1) Power method to find the eigenvalue. Involves matrix.vector iterations, and that can be broken into pieces. (2) Divide-and-conquer null space on matrix-eibval*identity. See this post. Weirdly enough, after never referencing I have now sent it to two people today. Also (3) remember what @Leonid Shifrin said: if you go to a low precision representation you will be asking for numerical troubles. –  Daniel Lichtblau Apr 4 '13 at 14:51

Matrices are best stored by Mathematica in packed arrayes. Packed arrays are limited to certain types. According to the docs‌​: "Supported types are Integer, Real, and Complex. "
Any change to SetPrecision practically applies a tenfold increase in bytes, because SetPrecision[..., small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking.