Why does memory run out when adding same-sized matrices in a loop?

I have written a module that it generates a 101x101x51 zero matrix at first. At every iteration, it generates a list by funct1 and based on this list, if list is empty it gives the previous result. If list is non-empty, it applies funct2 and get the same-sized matrix and add it to the previous one. funct2 uses BinCounts so it generates a matrix with scalar values in it..

The piece of code is :

resultfunction[minx_, miny_, minz_, maxx_, maxy_, maxz_] := Module[{points, i, j, k},
mtrx = Table[0, {101}, {101}, {51}];
Do[points = funct1[i, j, k];
If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)],
{i, minx, maxx}, {j, miny, maxy}, {k, minz, maxz}];
points =.;
mtrx]


The problem is that I am using a device with 32gb RAM, however it runs out of memory when I ran this module. I thought because of the fact that maybe it can be storing the points, at every iteration, I cleared up the points. However, the result is the same. When adding matrices, don't they use the same memory?

p.s. minx=miny=minz=1; maxx= 85, maxy=80, maxz=50

Any help is appreciated.

EDIT

funct1 is too long to write it here and it just return Null or a list whose dimensions are {length of the list,3}. funct2 is :

funct2[coefficient_, points_] :=
Block[{bsplinefunct, sampledpoints, results},
bsplinefunct = BSplineFunction@points;
results = funct3[20, 20, 5, 40, 40, 15, coefficient, {sampledpoints}];
{bsplinefunct, sampledpoints} =.;
results]


funct3 is :

funct3[minx_, miny_, minz_, maxx_, maxy_, maxz_, step_, points_] :=
Module[{matrix},
matrix = Total[Unitize[BinCounts[#, {minx, maxx + step, step}, {miny, maxy + step,
step}, {minz, maxz + step, step}]] & /@ points]]

• It is hard to say without seeing your funct1 and funct2. One possible guess is that some of these functions unpack (returned not packed, but unpacked array) - unpacked arrays can take up much more memory. It could also be that the huge memory consumption happens inside those functions. Feb 18, 2013 at 11:15
• I think that If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)] equivalent to If[Not[points === Null], mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)]. But I am not sure I can suggest anything relevant without funct1 and funct2
– acl
Feb 18, 2013 at 11:16
• @LeonidShifrin I haven't use a packed array yet so I don't know anything about it. Also, please see my edit.
– cesm
Feb 18, 2013 at 11:49
• You could add a \$HistoryLength=0 and should check if everything is a packed array.
– user21
Feb 18, 2013 at 18:44
• Try to reduce the code to a simple example anybody can reproduce. Then perhaps somebody will be interested enough to help. Feb 19, 2013 at 8:47

It is hard to say without seeing your actual functions, but you might want to consider refactoring your approach not to use a Do loop and not to overwrite your original matrix. Consider that what you are doing is getting the result of a function, and if that is a list and not null, adding some function of that list to a matrix. You don't care about the intermediate values of the matrix, and the value of the matrix to be added does not depend on the previous value of the matrix.

I am not near a Mathematica installation right now so my syntax might be wrong but why not try:

Plus @@ (funct2[0.2,#]& /@
Cases[Flatten[Table[funct1[i,j,k],(*iterators need not be whole set*)],1],_List])


You can break the problem up into several bits for different subsets of the desired arguments to funct1, and then add the resulting matrices together. Divide and conquer.