0
$\begingroup$

A comparison of float calculation and exact expression for

  NumberForm[0.3^2 - (3/10)^2, 18]

gives an exact zero for $0.3$. However,

R[n_Integer] = n 1./10.;
Rex[n_Integer] = n/10;
Table[R[n]^2 - Rex[n]^2, {n, 1, 5}]

gives

{1.73472*10^-18, 6.93889*10^-18, 2.77556*10^-17, 2.77556*10^-17, 0.}

Thus, here we get

0.3^2 - (3/10)^2 = 2.77556*10^-17

instead of zero. Why is the ouput different in the Table[] calculation?

$\endgroup$
  • $\begingroup$ Is your question: why does 0.3^2 - (3/10)^2 return 0. while R[3]^2 - Rex[3]^2 returns 2.77556*10^-17? $\endgroup$ – anderstood Jul 20 '17 at 18:06
  • $\begingroup$ Related, maybe a duplicate: mathematica.stackexchange.com/questions/135636/… $\endgroup$ – Michael E2 Jul 20 '17 at 18:30
  • $\begingroup$ This could be simplified considerably. It amounts to having done some arithmetic just to get the ".3" in R[3]: In[17]:= {.3, (3*1./10)} // InputForm Out[17]//InputForm= {0.3, 0.30000000000000004} $\endgroup$ – Daniel Lichtblau Jul 20 '17 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.