The compile function I have written computes a table of relative frequencies of some data. Thus, the output is a list of increase elements, p, in [0,1]. Since I have say 1000 data points the elements p are equal to x/1000 and x in [0,1000]. Hence, each p should have a maximum of three positions after the decimal point. However, if I run the compile function then values having many more positions after the decimal point appear. However, this appears to be true not for all elements of the resulting list.
For example, let the true p equal 0.894, then the compile output reads p = 0.8949999999999.
How can I force the result to be 0.894?
Unfortunately, Round[p,1./1000.]
does not work within the compile function. It does not work either if I apply Round
to the resulting list
. What works, but seem inefficient, is Round[list,1/1000]//N
. Thanks for help.
RealDigits[894/1000, 2]
will help you see what's going on: 0.894 has a nontrivial infinitely repeating expansion in base 2 (exactly like, say, 1/3 or 1/7 have repeating expansions base 10), proving that it's impossible to represent 0.894 exactly in base 2 floating point representations. $\endgroup$