1
$\begingroup$

I run these lines:

a = 0.833
SetPrecision[a, 20]

and this is the output:

0.833

0.83299999999999996270

I expected to receive

0.83300000000000000000

Do you have any suggestion? Thank you!

$\endgroup$
  • 2
    $\begingroup$ 0.833'20 gives what you expect, but SetPrecision[a,20] is equivalent to N[FromDigits[RealDigits[a,2],2],20] = 0.83299999999999996270. MMA works with binary numbers. $\endgroup$ – Alx Aug 25 '19 at 16:40
  • $\begingroup$ It works with 0.833'20, but not with a'20 $\endgroup$ – Andrea2810 Aug 25 '19 at 17:59
  • $\begingroup$ The back tick is used in specifying numeric input. Using it on variables thus a`20 is not syntactically valid. The problem is that a = 0.833 evaluates 0.833 in machine precision binary and obtains the binary floating point number nearest to 0.833, but that number is only approximately equal to 0.833. $\endgroup$ – Michael E2 Aug 25 '19 at 20:34
  • $\begingroup$ Related: mathematica.stackexchange.com/questions/55292/… $\endgroup$ – Michael E2 Aug 25 '19 at 20:38
  • 1
    $\begingroup$ You don't need a'20. Just use a=0.833'20 and a will have 20 digits of precision from then on or until it is reassigned. That is supposed to be a back tick, but I can't write it that way because of the way comments treat back ticks. $\endgroup$ – Bill Watts Aug 25 '19 at 21:58
1
$\begingroup$

I suggest SetPrecision[833/1000 , 20] or SetPrecision[.833 // Rationalize, 20].

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.