My program is eating up all my RAM and the operation is related to 3 matrices multiplication, as the code shown below.
example code:
{T, W, H} = {4, 10, 10}; (*crashed with larger values *)
matT = RandomReal[10, {T + 1, 2 T + 1}];
data = RandomReal[10, {W, H}];
matX = Table[RandomReal[i]*data, {i, 0, T}];
matX // Dimensions
matY = Table[RandomComplex[n+n I]*data, {n, -T, T}];
matY // Dimensions
Total[matX matT.matY]
After research I have detected the line of code which is killing a great deal of memory:
Total[matX matT.matY]
Where the dimension of matX
is {T+1,W,H}
, matY
: {2T+1,W,H}
and matT: {T+1,2T+1}
, T
is any integer but preferred even numbers.
edit:
one of the reason the RAM is being eaten up is due to the fact matY
has imaginary part which is obtained from Fourier angular functions. The byte count is double size of real counterpart matX
which only has real numbers.
This code works fine when {T,W,H}
are small. However,my goal is to calculate the case when {T,W,H}={1000,1000,1000}
, but now I only half way to it.
I guess MMA is copying a lot of data in the RAM to do this Total
calculation and I think it can be improved in terms of reducing RAM usage. I would really appreciate your helps to give me some ideas.
matX
,matY
, andmatT
come from? Maybe that structure can be exploited. And what are the typical valued ofT
,W
, andH
that you want to use in practice? $\endgroup${W,H }
cannot be any less while I can splitT
dimensions into many parts. It's only solution I have now. It's a bit of tedious but it works anyway. $\endgroup$RandomComplex[n + n I]
and notRandomComplex[n + n i]
$\endgroup$