# Table size/dimension error

I'm trying to create a table of the form

n=20;
V = Table[
0, {b1, 1, n}, {a1, 1, n}, {b2, 1, n}, {a2, 1, n}, {b3, 1, n}, {a3,
1, n}, {b4, 1, n}, {a4, 1, n}];


but it fails with a beep after some time. Is there anyway to work with large arrays? I would like to use n>100 and go up to b100, a100 or as close as possible.

• Please consider ConstantArray[0, {n, n, n, n, n, n, n, n}, SparseArray] which won't consume vast amounts of memory for just storing zeroes. Aug 11, 2022 at 18:36

Introductory example. Consider a $$20 \times 20 \times 20 \times 20$$ array:

A = Table[0,{20},{20},{20},{20}];


How much memory does it use?

ByteCount[A]
(* 1280224 *)


We see that it uses a little more than 1 megabyte. It is easy to understand where this number comes from: On a 64 bit computer, one number uses 8 byte, and therefore $$20^4$$ numbers use $$20^4 \cdot 8 = 1280000$$ bytes. The extra $$224$$ byte can be thought of as a bookkeeping overhead.

Generalization. An array with dimensions $$\underbrace{n \times \cdots \times n}_{d}$$ will use $$n^d \cdot 8$$ bytes. Let us consider two examples:

• In the question, OP uses $$n=20$$ and $$d=8$$, which gives $$20^8 \cdot 8 = 204800000000$$ bytes which is 205 gigabyte. That is not an outrageous amount of memory, but more than the memory of a typical personal computer. Hence, unsurprisingly, the request is not carried out.

• OP expresses interest in $$n=100$$ and $$d = 200$$, which gives $$100^{200} \cdot 8 \approx 10^{400}$$ bytes and which is an outrageous amount of memory. This is not realistic.

Disclaimer. The counting I used above is simplified and assumes that the array is stored in a compact form. In Mathematica this is known as a "packed array". The array A in the introductory example is packed, as can be checked using

A // DeveloperPackedArrayQ
(* True *)


Here is an example that is not packed and uses more memory than the simplified counting would suggest:

Table[i^j*k^l,{i,1,20},{j,1,20},{k,1,20},{l,1,20}]//ByteCount
(* 7045544 *)


The reason is that individual entries of this array are large and use up more than 8 byte.

• Mathematica also has SparseArray`s which cut storing default values (usually zeroes), but it is a bit unclear what use this $100^8$ array would actually have - for very sparse data structures, one could also use memoization, for instance. I don't think there even is a shared-memory HPC system with an order of 100 petabytes of efficiently addressable memory which would be required for non-sparse approach. Aug 12, 2022 at 8:27