0
$\begingroup$

I have written a module that it generates a 101x101x51 zero matrix at first. At every iteration, it generates a list by funct1 and based on this list, if list is empty it gives the previous result. If list is non-empty, it applies funct2 and get the same-sized matrix and add it to the previous one. funct2 uses BinCounts so it generates a matrix with scalar values in it..

The piece of code is :

resultfunction[minx_, miny_, minz_, maxx_, maxy_, maxz_] := Module[{points, i, j, k},
mtrx = Table[0, {101}, {101}, {51}];
Do[points = funct1[i, j, k];
If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)],
{i, minx, maxx}, {j, miny, maxy}, {k, minz, maxz}];
points =.;
mtrx]

The problem is that I am using a device with 32gb RAM, however it runs out of memory when I ran this module. I thought because of the fact that maybe it can be storing the points, at every iteration, I cleared up the points. However, the result is the same. When adding matrices, don't they use the same memory?

p.s. minx=miny=minz=1; maxx= 85, maxy=80, maxz=50

Any help is appreciated.

EDIT

funct1 is too long to write it here and it just return Null or a list whose dimensions are {length of the list,3}. funct2 is :

funct2[coefficient_, points_] := 
Block[{bsplinefunct, sampledpoints, results},
bsplinefunct = BSplineFunction@points;
sampledpoints = Thread[bsplinefunct[Range[0, 1, .0001]]];
 results = funct3[20, 20, 5, 40, 40, 15, coefficient, {sampledpoints}];
{bsplinefunct, sampledpoints} =.;
results]

funct3 is :

funct3[minx_, miny_, minz_, maxx_, maxy_, maxz_, step_, points_] := 
Module[{matrix}, 
matrix = Total[Unitize[BinCounts[#, {minx, maxx + step, step}, {miny, maxy + step, 
     step}, {minz, maxz + step, step}]] & /@ points]]
Improve this question
$\endgroup$
10
  • $\begingroup$ It is hard to say without seeing your funct1 and funct2. One possible guess is that some of these functions unpack (returned not packed, but unpacked array) - unpacked arrays can take up much more memory. It could also be that the huge memory consumption happens inside those functions. $\endgroup$
    – Leonid Shifrin
    Commented Feb 18, 2013 at 11:15
  • $\begingroup$ I think that If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)] equivalent to If[Not[points === Null], mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)]. But I am not sure I can suggest anything relevant without funct1 and funct2 $\endgroup$
    – acl
    Commented Feb 18, 2013 at 11:16
  • $\begingroup$ @LeonidShifrin I haven't use a packed array yet so I don't know anything about it. Also, please see my edit. $\endgroup$
    – cesm
    Commented Feb 18, 2013 at 11:49
  • 1
    $\begingroup$ You could add a $HistoryLength=0 and should check if everything is a packed array. $\endgroup$
    – user21
    Commented Feb 18, 2013 at 18:44
  • 2
    $\begingroup$ Try to reduce the code to a simple example anybody can reproduce. Then perhaps somebody will be interested enough to help. $\endgroup$
    – Ajasja
    Commented Feb 19, 2013 at 8:47

1 Answer 1

Reset to default
2
$\begingroup$

It is hard to say without seeing your actual functions, but you might want to consider refactoring your approach not to use a Do loop and not to overwrite your original matrix. Consider that what you are doing is getting the result of a function, and if that is a list and not null, adding some function of that list to a matrix. You don't care about the intermediate values of the matrix, and the value of the matrix to be added does not depend on the previous value of the matrix.

I am not near a Mathematica installation right now so my syntax might be wrong but why not try:

Plus @@ (funct2[0.2,#]& /@ 
 Cases[Flatten[Table[funct1[i,j,k],(*iterators need not be whole set*)],1],_List])

You can break the problem up into several bits for different subsets of the desired arguments to funct1, and then add the resulting matrices together. Divide and conquer.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.