Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have written a module that it generates a 101x101x51 zero matrix at first. At every iteration, it generates a list by funct1 and based on this list, if list is empty it gives the previous result. If list is non-empty, it applies funct2 and get the same-sized matrix and add it to the previous one. funct2 uses BinCounts so it generates a matrix with scalar values in it..

The piece of code is :

resultfunction[minx_, miny_, minz_, maxx_, maxy_, maxz_] := Module[{points, i, j, k},
mtrx = Table[0, {101}, {101}, {51}];
Do[points = funct1[i, j, k];
If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)],
{i, minx, maxx}, {j, miny, maxy}, {k, minz, maxz}];
points =.;
mtrx]

The problem is that I am using a device with 32gb RAM, however it runs out of memory when I ran this module. I thought because of the fact that maybe it can be storing the points, at every iteration, I cleared up the points. However, the result is the same. When adding matrices, don't they use the same memory?

p.s. minx=miny=minz=1; maxx= 85, maxy=80, maxz=50

Any help is appreciated.

EDIT

funct1 is too long to write it here and it just return Null or a list whose dimensions are {length of the list,3}. funct2 is :

funct2[coefficient_, points_] := 
Block[{bsplinefunct, sampledpoints, results},
bsplinefunct = BSplineFunction@points;
sampledpoints = Thread[bsplinefunct[Range[0, 1, .0001]]];
 results = funct3[20, 20, 5, 40, 40, 15, coefficient, {sampledpoints}];
{bsplinefunct, sampledpoints} =.;
results]

funct3 is :

funct3[minx_, miny_, minz_, maxx_, maxy_, maxz_, step_, points_] := 
Module[{matrix}, 
matrix = Total[Unitize[BinCounts[#, {minx, maxx + step, step}, {miny, maxy + step, 
     step}, {minz, maxz + step, step}]] & /@ points]]
share|improve this question
    
It is hard to say without seeing your funct1 and funct2. One possible guess is that some of these functions unpack (returned not packed, but unpacked array) - unpacked arrays can take up much more memory. It could also be that the huge memory consumption happens inside those functions. –  Leonid Shifrin Feb 18 '13 at 11:15
    
I think that If[points === Null, mtrx, mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)] equivalent to If[Not[points === Null], mtrx = mtrx + funct2[0.2, points](*0.2 is a coefficient*)]. But I am not sure I can suggest anything relevant without funct1 and funct2 –  acl Feb 18 '13 at 11:16
    
@LeonidShifrin I haven't use a packed array yet so I don't know anything about it. Also, please see my edit. –  cesm Feb 18 '13 at 11:49
1  
You could add a $HistoryLength=0 and should check if everything is a packed array. –  user21 Feb 18 '13 at 18:44
2  
Try to reduce the code to a simple example anybody can reproduce. Then perhaps somebody will be interested enough to help. –  Ajasja Feb 19 '13 at 8:47
show 5 more comments

1 Answer

It is hard to say without seeing your actual functions, but you might want to consider refactoring your approach not to use a Do loop and not to overwrite your original matrix. Consider that what you are doing is getting the result of a function, and if that is a list and not null, adding some function of that list to a matrix. You don't care about the intermediate values of the matrix, and the value of the matrix to be added does not depend on the previous value of the matrix.

I am not near a Mathematica installation right now so my syntax might be wrong but why not try:

Plus @@ (funct2[0.2,#]& /@ 
 Cases[Flatten[Table[funct1[i,j,k],(*iterators need not be whole set*)],1],_List])

You can break the problem up into several bits for different subsets of the desired arguments to funct1, and then add the resulting matrices together. Divide and conquer.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.