I've never seen this error on such a simple calculation. Does anybody know how to fix this annoying error?

A very simple calculation causes the max precision limit to overflow

  • $\begingroup$ Just noticed it's the a-b that's causing this, when I remove it, it doesn't require a high extra precision anymore. $\endgroup$ – user58873 Jul 8 '18 at 19:15
  • 1
    $\begingroup$ If a is equal to b, and try and calculate N[a-b, anything], you'll get that exact same error, no matter how much you increase the $MaxExtraPrecision. $\endgroup$ – AccidentalFourierTransform Jul 8 '18 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy