11
$\begingroup$

The answer to this question will most probably be a definite no. Nevertheless, I would like to know whether it is possible to represent reals in the memory on less than the standard 32 or 64 bits (depending on machine). This limits my exploration to $\sim 2^{14} \times2^{14}$ matrices, though with half or quarter precision I could venture a bit further. The matrix itself cannot be broken down to smaller pieces for further compuation, and it cannot be reworked to a simpler one, but high precision is of no importance. The question is not about refactoring matrices but about number representation and memory handling by Mathematica.

Warning, the following computation may eat through your memory limits.

n = 14;
all = (2^n)*(2^n);
m = RandomReal[{0, 1}, {2^n, 2^n}];
{all, (ByteCount@m/1024./1024) "MB", ((ByteCount@m/all*8.) // N) "bit"}
ClearAll[m];

{268435456, 1024. "MB", 64. "bit"}

I am using 64-bit Win7, with 2 GB RAM, and (expectedly) my system almost chokes on this code. And then of course I hadn't calculated anything with the matrix yet...

Improve this question
$\endgroup$
11
  • 2
    $\begingroup$ High precision may be of more importance than it appears, since for large matrices, at least in some cases, one may get large accumulated errors in the computations - this of course depends on the matrix and on the operations you want to do with it. $\endgroup$
    – Leonid Shifrin
    Apr 4, 2013 at 9:47
  • $\begingroup$ @Leonid Yes, I am aware of that, but still I wonder if it is possible to represent my data on less then the standard amount of memory. Or should I switch to C (or similar) if I want half precision... $\endgroup$
    – István Zachar
    Apr 4, 2013 at 9:49
  • $\begingroup$ You could give SetPrecision a try. But for a single float, this increases the memory usage from 24byte to 72byte. $\endgroup$
    – Stefan
    Apr 4, 2013 at 9:51
  • 2
    $\begingroup$ SetPrecision[...,something small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking. $\endgroup$
    – Daniel Lichtblau
    Apr 4, 2013 at 13:36
  • 2
    $\begingroup$ Try the following using out-of-core memory. (1) Power method to find the eigenvalue. Involves matrix.vector iterations, and that can be broken into pieces. (2) Divide-and-conquer null space on matrix-eibval*identity. See this post. Weirdly enough, after never referencing I have now sent it to two people today. Also (3) remember what @Leonid Shifrin said: if you go to a low precision representation you will be asking for numerical troubles. $\endgroup$
    – Daniel Lichtblau
    Apr 4, 2013 at 14:51

1 Answer 1

Reset to default
4
$\begingroup$

(Based on the comments)

Matrices are best stored by Mathematica in packed arrayes. Packed arrays are limited to certain types. According to the docs‌​: "Supported types are Integer, Real, and Complex. "

Any change to SetPrecision practically applies a tenfold increase in bytes, because SetPrecision[..., small] will cause machine numbers, which might live in a packed array, to become low precision, unpackable, bignums. Big loss in terms of memory due to the unpacking.

Just for the record, IEEE 754-2008 does allow 16-bit floats (Half-precision floating-point format) so it is possible to lower bitwidth in e.g. C, however do take into account the possibility that with lower precision, accumulated error might be much higher in some computations.

Improve this answer
$\endgroup$
1
  • $\begingroup$ It is a real shame, that mathematica does not allow that. The standard seems to be 64bit floats by now and that is way more than you need in most cases and hence costs a lot of unnecessary memory. $\endgroup$
    – Wizard
    Jun 6, 2015 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.