The compile function I have written computes a table of relative frequencies of some data. Thus, the output is a list of increase elements, p, in [0,1]. Since I have say 1000 data points the elements p are equal to x/1000 and x in [0,1000]. Hence, each p should have a maximum of three positions after the decimal point. However, if I run the compile function then values having many more positions after the decimal point appear. However, this appears to be true not for all elements of the resulting list.
For example, let the true p equal 0.894, then the compile output reads p = 0.8949999999999.
How can I force the result to be 0.894?
Round[p,1./1000.]does not work within the compile function. It does not work either if I apply
Roundto the resulting
list. What works, but seem inefficient, is
Round[list,1/1000]//N. Thanks for help.