# Is this caused by meomory usage ? (Something odd/hard to describe)

I really can't make a "smaller" example, but the files are here, for the data and for the notebook. It will cost a LOT of memory, up to 30G of RAM (maybe more).

Here is the problem, (from a Linux Server, MMA10.0.2 X64)

It does look like it finished the job. But res did not save as expected. I think this rise from a memory problem? Because at some point, the server(system) hang for a while.

(res = ParallelTable[
myclosepop[dat[indat], kdat[indat], m], {indat, data}, {m, models}
];) // AbsoluteTiming


res will finish and save without any problem. The memory usage during the calculation will also be "smaller".

My main question is:

1 - can the code be improved to avoid this? 2 - What does the error message tell me?? Join::heads:...... does really not tell me much. I did not use it explicitly. And I dont even know whether this is a memory related problem.

Thanks!

Update: Individual ones do run OK, say with a smaller subsets of the models

I am doing in this way, hoping to get a "nice" table, with all results in it.

• What happens if you run a subset of the data/models, or use Table instead of ParallelTable? – 2012rcampion Jan 29 '15 at 23:30
• Single ones do run fine, even with different methods and search points ect. But I want to make a nice table like this, say run models = {"Bin", "BetaBin"}; then the res = .... then {lmax, aics, mles} = Transpose[res, {2, 3, 1}]; TableForm[lmax, TableHeadings -> {data, models}]. These gives an informative table, see updated post. – Chen Stats Yu Jan 30 '15 at 1:04

My guess is you are running into the same issue as was reported here. A possible workaround is to give NMinimize an explicit PostProcess option value, e.g."PostProcess" -> {"KKT"} (this goes inside the Method option setting).

• I will try and see if that works. I have also updated the post, explaining why I am doing this. I will report the result back tomorrow. – Chen Stats Yu Jan 30 '15 at 1:07
• I think the same problem still occur. On the server, the memory usage quickly goes up to about 40G and the process become a status "D" (it hangs basically). – Chen Stats Yu Jan 30 '15 at 14:58