# Floor inconsistent with Less for machine-precision approximate numbers

For machine-precision numbers, Mathematica uses a tolerance for comparisons, so that 1.-$MachineEpsilon==1. However, Floor does not take into account this tolerance, leading to an inconsistency: {Floor[#], # < 1., # >= 1.} &[1. -$MachineEpsilon]
(*{0, False, True}*)
(*Should be {1,False,True} or {0,True,False}*)


For the particular piece of code this affects, I am using Compile and so can just set the RuntimeOptions to not CompareWithTolerance (which is what Floor assumes). However, I thought it worth asking if it is possible to avoid this in general, or is it necessary to refactor code that assumes this consistency?

For what it's worth, it also appears to be inconsistent with the documentation which states that "Floor[x] gives the greatest integer less than or equal to x" and the documentation gives an example of it working correctly for an arbitrary-precision number, e.g. Floor[1100 - 10^-130]==1

• In the documentation for Less (and equal, etc.) It specifies that it does not consider the last bit in the comparison, as you're seeing here. Also, the example you list: Floor[1100 - 10^-130]==1 is listed under "Possible Issues", which indicates that it doesn't necessarily work as expected; this is because you're specifying a precision of 100 digits with 1100, whereas 1. is specified as MachinePrecision; clearly 10^-130 would require more than 130 digits of precision, and so is neglected in evaluation. – Michael Witt Jul 9 '15 at 1:40
• @Michael, that's almost an answer already. :) Consider writing one when you find the time. – J. M. will be back soon Jul 9 '15 at 5:26
• @MichaelWitt if I understand the documentation correctly, it is the last byte that is ignored by Less. This doesn't particularly bother me. However, the definition of equality should be consistent throughout. The greatest integer less than or equal to 1.-$MachineEpsilon is 1 according to Less and 0 according to Floor. It gives a bug in code such as If[1<=x<Length[list]+1,list[[Floor[x]]],0]. If x is 1.-$MachineEpsilon, we get Head[list] rather than 0 or list[] as expected. – Joel Bosveld Jul 10 '15 at 2:29

The explanation is pretty much the same as the one given in Bug in floating-point number comparisons near $MachineEpsilon? The OP seems familiar with the reasoning given there. For a workaround, as explained in the linked Q&A, one can do the following to set the tolerance to zero: Block[{Internal$EqualTolerance = 0},

As for the second example, one can see that 1100 - 10^-130 evaluates to 1.100.; consequently, Floor[] will evaluate to 1.
1100 - 10^-130 // InputForm