Tag Info

New answers tagged

4

The documentation is misleading in this case. The wrapper NumberForm is not transparent to numerical calculation -- it stops it dead. This means you should only apply it to your final calculations when all the numcerical work is done. So the following works. NumberForm[{n1, n2 - n1}, 3] {0.235, 0.766} If what you want is for all the numerical work in ...


4

Already answered in the comments by DumpsterDoofus and Daniel Lichtblau, to summarize: Machine floating point numbers such as 0.2 are not always exactly representable in binary (no terminating expansion in base 2). Thus floating point arithmetic is susceptible to roundoff error and other accuracy problems. For example, the following are not exactly equal to ...



Top 50 recent answers are included