2
$\begingroup$

I need to solve about a million of QP problems, each having about N=2000 variables with N-2 inequality constraints and 2 equality constraints.

Trying to use the "InteriorPoint" method freezes my Computer for several minutes, and then crashes the kernel. "IPOPT" method gives an error that the computation can not continue since there is not enough memory (it seems like 16GB of RAM is not enough).

The undocumented "QuadraticProgramming" method solves each QP in less than a second (while the other methods took hundreds of times more even with smaller values of N where they don't crash).

The problem is that calling FindMinimum with that method randomly crashes the kernel after being called a few thousand times.

I tried generating a task with ParallelSubmit so FindMinimum crashes a subkernel instead of the main kernel, which seems to work for not losing the results obtained so far, but after there is a crash, Mathematica doesn't seem to be able to start a new subkernel anymore (it takes forever to start a new one, causing all retries to fail until I restart the main kernel again).

The relevant code: (Objective, Constraints and YFitStart are local variables inside a Module clause, saveme saves the current results so far in a file using a DumpSave call)

SakeTask:=(If[$KernelCount<1,LaunchKernels[1]];
SetSharedVariable[Objective,Constraints,YFitStart];
task=ParallelSubmit[FindArgMin[Join[{Objective},Constraints],
YFitStart,Method->"QuadraticProgramming",MaxIterations->Infinity]];
task=WaitAll[task];
UnsetShared[Objective,Constraints,YFitStart];
task);

rtask=$Failed;
While[rtask==$Failed,
rtask=TimeConstrained[SakeTask,100,Print["Oh no"];saveme;CloseKernels[];$Failed];
];
$\endgroup$
  • $\begingroup$ Can you try an experiment? Write a batch file of 5000 lines. Each line invokes MMA, supplies the input needed to solve the next QP proroblem and redirects the output to append to a log file. MMA does one problem and then exits. For example: on the desktop in.txt containing Print[2+3], out.txt containing nothing, try.bat containing "C:\Program Files\Wolfram Research\Mathematica\11.1\math" -noprompt <in.txt >>out.txt Execute that bat and out.txt then contains 5 and MMA has terminated. See if your script can succeed with 5000 separate single problems. $\endgroup$ – Bill Sep 27 '17 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.