3
$\begingroup$

I have created a table Tv where each row is of the form

{x, y, Iv[x,y]}

where Iv is some complicated numerical integral and x and y iterate through some parameter space. The issue I am facing now is that for some values of x and y, Iv[x,y] failed to evaluate. Since these are only a few cases, I would like to continue working with the rest of the data. Unfortunately, however, when I try to print or save Tv, mathematica tries to re-evaluate the cells that failed before, which takes many hours (and, of course, they fail again).

So my question is: Can I remove all rows of Tv with non-numeric entries in the third column without re-evaluating these cells?

Edit:

Here is integral that gives me troubles:

kernel[x_, xp_, p_, d_] := 
 2 Sqrt[x^2 + xp^2 - 2 x xp Cos[p]] - 
  2 Sqrt[d^-2 + x^2 + xp^2 - 2 x xp Cos[p]] - 
  1/2 d^-1 (Log[x^2 + xp^2 - 2 x xp Cos[p]] + 
     Log[-d^-1 + Sqrt[d^-2 + x^2 + xp^2 - 2 x xp Cos[p]]] - 
     3 Log[d^-1 + Sqrt[d^-2 + x^2 + xp^2 - 2 x xp Cos[p]]])
rdivM[x_, x0_] := 
 x (-1 + (4 Sinh[x]^2)/(Cosh[2 x] + Cosh[2 x0])) (Sech[x - x0] + 
     Sech[x + x0]) + 4 Cosh[x0] Sinh[x]/(Cosh[2 x] + Cosh[2 x0])
Iv[x0_, d_] := 
 2 NIntegrate[
   rdivM[x, x0] rdivM[xp, x0] kernel[x, xp, p, d], {x, 
    Max[0, x0 - 50], x0 + 50}, {xp, Max[0, x0 - 50], x}, {p, 0, Pi}, 
   PrecisionGoal -> 6, 
   Method -> {"GlobalAdaptive", "MaxErrorIncreases" -> 100000}]

and the evaluation:

x0List = {0, 0.1, 0.2, 0.3, 0.5, 0.7, 1, 1.5, 2, 2.5, 3, 4, 5, 7, 10, 
   15, 20, 35, 50, 75, 100, 150, 200};
logdList = Range[-4, 4, 1/10];
Tdv = ConstantArray[0, {Length[x0List]*Length[logdList], 3}];
SetSharedVariable[Tdv];
n = 0;
SetSharedVariable[n];
ParallelDo[Tdv[[d + r Length[logdList] + 1, 1]] = x0List[[r + 1]];
 Tdv[[d + r Length[logdList] + 1, 2]] = N[10^logdList[[d + 1]]];
 Tdv[[d + r Length[logdList] + 1, 3]] = 
  Re[Iv[N[x0List[[r + 1]]], N[10^logdList[[d + 1]]]]]; n++; 
 Print[n], {r, 0, Length[x0List] - 1}, {d, 0, Length[logdList] - 1}]

The evaluation will take many hours, but you can abort after a the first error messages. Then just try to print Tdv.

$\endgroup$
  • $\begingroup$ You can probably use a combination of If[] and Nothing for this. $\endgroup$ – J. M. will be back soon Jun 7 '16 at 16:21
  • $\begingroup$ How would I do this without re-evaluating the cells? So far, every operation on Tv has lead to a re-evaluation of failed cells. $\endgroup$ – Felix Jun 7 '16 at 16:56
  • $\begingroup$ You may have to re-factor the structure of your calculation to avoid shared variables in parallel calculations. See: Side effects with ParallelDo and Why won't Parallelize speed up my code?. $\endgroup$ – MarcoB Jun 7 '16 at 20:23
  • $\begingroup$ I have no problem with the performance of the ParallelDo[...] code. Also, I don't use Append[...] and the data that is evaluated seems to be all consistent. The problem is only about the fraction of the data that fails to calculate (it also fails when I evaluate it outside the ParallelDo[...] loop). $\endgroup$ – Felix Jun 7 '16 at 20:46
  • $\begingroup$ @Felix, take a look at a different approach I show in a new answer below. $\endgroup$ – MarcoB Jun 7 '16 at 23:52
4
$\begingroup$

Wrap the call to the function that is likely to generate errors and return unevaluated in a Check expression. In the case of correct execution, Check will pass through the value returned by its argument; in case of an error, it will return an expression of your choice.

This "failure indicator" won't trigger recalculation, and can be made very recognizable so it will be easy to remove data points that contain it. Two such examples could be $Failed or Missing[]. I prefer the latter, since it naturally works well with the DeleteMissing command.

In short, I propose the following modification in your ParallelDo code:

Tdv[[d + r Length[logdList] + 1, 3]] = 
  Check[
    Re[Iv[N[x0List[[r + 1]]], N[10^logdList[[d + 1]]]]], 
    Missing[], 
    {Infinity::indet, NIntegrate::inumri}
  ];

In those cases where errors as Infinity::indet or NIntegrate::inumri (i.e. the integrand has evaluated to Overflow, Indeterminate, or Infinity for all sampling points in the region) are generated by the calculation, Missing[] will be returned.

Here is a sample obtained from running the modified code for a while, then aborting it, and evaluating Tdv:

{ {0, 0.0001, Missing[]}, {0, 0.000125893, 6.11017*10^6}, 
  {0, 0.000158489, 4.7075*10^6}, .... , {0, 0, 0}, {0, 0, 0}, 
  {0, 0, 0}, {1, 0.0001, Missing[]}, {1, 0.000125893, 7.75528*10^6}, 
  {1, 0.000158489, 5.9716*10^6}, .... }

Since no unevaluated NIntegrate expression was left behind, no re-evaluation was triggered.

You can then remove the "bad" points using DeleteMissing with an appropriate level specification:

DeleteMissing[Tdv, 1, Infinity]

{ {0, 0.000125893, 6.11017*10^6}, 
  {0, 0.000158489, 4.7075*10^6}, .... , {0, 0, 0}, {0, 0, 0}, 
  {0, 0, 0}, {1, 0.000125893, 7.75528*10^6}, 
  {1, 0.000158489, 5.9716*10^6}, .... }

Following up on your comment looking for a way to prevent re-evaluation "after the fact", i.e. on a list of results that were generated without the Check, you can achieve that as well.

The simplest way I can think of is to temporarily redefine the function whose evaluation you want to prevent. You can do that with a local definition within the scope of a Block, as follows (here Tdv is the result of a partial evaluation as used above):

Block[{NIntegrate = Inactive[NIntegrate]},
  Tdv
]

inactivated results

This effectively prevents evaluation of NIntegrate so wastes no time.

You can also use this to filter out the unwanted unevaluated results:

cleanTdv = Block[{NIntegrate = Inactive[NIntegrate]},
  Cases[Tdv, {_, _, _?NumericQ}]
]

no NIntegrate any more

$\endgroup$
  • $\begingroup$ That is a very helpful approach. Following it, I now recalculated the entire table Tdv. Still, out of curiosity: Would it have been possible to apply this solution to the table after its evaluation with the original code? Like, if one happens to run the code without the Check function for some time, is there a way to recover the result without having to re-evaluate? $\endgroup$ – Felix Jun 8 '16 at 1:30
  • $\begingroup$ @Felix Yes you can do that as well. I have added such an approach as a second part to my answer above. $\endgroup$ – MarcoB Jun 8 '16 at 16:45
  • $\begingroup$ Perfect, exactly what I needed. $\endgroup$ – Felix Jun 8 '16 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.