Definition of $MachineEpsilon $MachineEpsilon
x = 0.0000000000000001
% + 1
% - 1


As expected the final result equals zero, since x < MachineEpsilon ($2.22045\times10^{-16}$).

However, if I double x (to 0.0000000000000002), then the final result should in my understanding still be zero, since x is still smaller than the MachineEpsilon, but... to my surprise the final result isn't zero! Question: why is this?

Note 1: If I triple x (to 0.0000000000000003), then x is larger than the MachineEpsilon and indeed the final result is bigger than zero.

Note 2: On my computer the value for MachineEpsilon is $2.22045\times10^{-16}$ Is this the same as on any computer using Mathematica by the way?

EDIT: I noticed that in binary form:

1. The MachineEpsilon is $1.\times2^{-52}$
2. And 0.0000000000000002 is $1.1100110100101011001\times2^{-53}$

Code:

BaseForm[$MachineEpsilon, 2] x = BaseForm[0.0000000000000002, 2] % + 1 BaseForm[% - 1, 2]  Therefore, could it be that$1.1100110100101011001\times2^{-53}$is rounded up to$1.\times2^{-52}$if$1$is added? Which is precisely equal to the MachineEpsilon and that therefore the final result above with 0.0000000000000002 isn't zero but... the MachineEpsilon!? • In the past $MachineEpsilon in Windows-based and OS-X-based systems differed because the Windows-based systems used Intel processors and Apple used Motorola/IBM processors. Apple changed over to Intel in 2006. So now it would be hard to find a system running Mathematica that doesn't give 2.220446049250313*^-16 for $MachineEpsilon because that is the value retuned by any system running an Intel Core i3, i5, or i7 processor(s). Sep 9, 2016 at 14:28 • Log[2,$MachineEpsilon] evaluates to -52. and RootApproximant[$MachineEpsilon] === 2^-52 evaluates to True Sep 9, 2016 at 15:19 4 Answers I suppose this is due to rounding. For as long as the value is smaller than half of $MachineEpsilon it yields 0, else it rounds up to $MachineEpsilon. Given x*i+1-1 For, i<0.5*10^15$MachineEpsilon I get zero,
while for i>0.5*10^15 $MachineEpsilon i get $MachineEpsilon.

And regarding your second question: On my laptop

$MachineEpsilon = 2.22045×10−16 It might be a naive way to show it, but here's an illustration. DiscretePlot[{(x*i) + 1 - 1}, {i, 1.1101, 1.11035, 0.000001}]  On a binary machine with$n$-bit floating point numbers, machine$\epsilon$is the smallest number such that$1 + \epsilon$can be represented in$n$bits. This is $$1.000\cdots01$$ where the length of$1$s and$0$s is$n$; in other words $$1 + \epsilon = 2^0+2^{-(n-1)}\,.$$ For the standard IEEE binary64 floating point number$n = 53$and $$\epsilon = 2^{-52}\,.$$ So that's the story for binary machines. When you start to investigate machine$\epsilon$on such a machine using decimal numbers, you have to consider rounding error and how arithmetic operations are carried out. Consider 1 * 10^-16 and 2 * 10^-16 in binary: BaseForm[0.0000000000000001, 2] BaseForm[0.0000000000000002, 2] (* 1.1100110100101011001 * 2^-54 1.1100110100101011001 * 2^-53 *)  Now an arithmetic operation is carried out by the CPU with extra guard bits, so the the result is equal to the exact result of the operation rounded to$nbits. So \eqalign{ 1. &+\ 2. * 10^{-16} = \cr &1.00000000000000000000000000000000000000000000000000001110011\cdots \cr } rounded to 53 bits on a binary64 machine, which yields, $$1.0000000000000000000000000000000000000000000000000001$$ If we compare them side by side, 1.00000000000000000000000000000000000000000000000000001110011... 1.0000000000000000000000000000000000000000000000000001  we see that the 54th & 55th bits cause the 53rd bit to be rounded up to 1, just as the OP suggested. Hence x = 0.0000000000000002 % + 1 % - 1  has a nonzero final result, namely 2.^-52. On the other hand, in1. + 1. * 10^{-16}$, there is another 0 between the first and second 1s, so that the 54th & up bits are 01.... Thus the 53rd bit is rounded to 0, and the result is 1.. Subtracting 1 from it yields zero, as in the OP's first example. What happens when adding 1 to a number close to MachineEpsilon? Some of the confusion in the question is caused by the way Mathematica displays machine-precision numbers. For example, Mathematica's "nice" default settings make it appear that there's no difference between 1.0 and 1.0 +$MachineEpsilon.

1.0
1.0 + $MachineEpsilon  The displayed results are the same, that is, 1.. However, we can use RealDigits to show that 1.0 +$MachineEpsilon is not the same as 1.0 at machine-precision (assuming MachinePrecision is $\frac{53 \log (2)}{\log (10)}$≈15.9546).

RealDigits[1.0, 2] (* binary digits *)
RealDigits[1.0 + $MachineEpsilon, 2]  The results are: {{1, 0, <<50 zeros>>, 0}, 1} (* least-significant digit is 0 *) {{1, 0, <<50 zeros>>, 1}, 1} (* least-significant digit is 1 *)  The least-significant binary digit shows that 1.0 and 1.0 +$MachineEpsilon are in fact different, and demonstrates that MachineEpsilon is "the minimum positive machine-precision number which can be added to 1.0 to give a result distinguishable from 1.0."

The point is that we can't rely on Mathematica's "nice" display of machine-precision numbers. We need to look as closely as possible at the binary result.

Case #1, where x = 0.0000000000000001

Consider the result of adding 1 to 0.0000000000000001.

x = 0.0000000000000001;
RealDigits[x + 1, 2]}


The result is {{1, 0, <<50 zeroes>> 0}, 1}, and notice the least-significant digit is 0.

Case #2, where x = 0.0000000000000002

x = 0.0000000000000002;
RealDigits[x + 1, 2]


This time, the result is {{1, 0, <<50 zeros>> 1}, 1}. Compare to case #1, where the least-significant digit was 0. This shows how decimal-to-binary conversion and using a number twice as large as case #1 produces a different result, and that the result is MachineEpsilon larger.

The different least-significant binary digits are the reason why the case #2 result isn't zero, and it is in fact the same as MachineEpsilon.

The case where x = 0.0000000000000003 is the same as x = 0.0000000000000002.

x = 0.0000000000000003;
RealDigits[x + 1, 2]


Another way to look at these results is to use extended-precision numbers instead of machine-precision values. For example, using case #2:

x = 0.0000000000000002;
SetPrecision[x + 1, 22]


The result is 1.000000000000000222045. Notice the the digits 222045 from the value of MachineEpsilon.

To summarize, Mathematica displays machine-precision numbers in a "nice" way that doesn't show the full precision of the result of a calculation. We need to use RealDigits or use extended precision to see the results of adding 1 to numbers close to MachineEpsilon.

Is MachineEpsilon the same as on any computer using Mathematica?

The value of MachineEpsilon is determined by the internal representation of machine-precision floating-point numbers. The value of MachinePrecision is $\frac{53 \log (2)}{\log (10)}$≈15.9546, when a computer uses IEEE Standard 754-1985 double-precision, 64-bit floating-point numbers. The value 53 comes from the number of bits used to store the mantissa (52) plus an assumed, normalized 1 bit to the left of the binary mantissa.

Seems like $MachinePrecision and $MachineEpsilon are inconsistent with each other.

Logically MachineEpsilon should be equal to $\frac{1}{10^{$MachinePrecision}}$but in fact MachineEpsilon =$ \frac{2}{10^{$MachinePrecision}}$

So in fact when you multiply your number by 2 you get a number that is larger than the smallest allowed number by MachinePrecision.

Also the threshold that @Dede found correspond to $MachineEpsilon/2 Which in fact the (effective $MachineEpsilon) = $\frac{1}{10^{$MachinePrecision}}$and not whatever $MachineEpsilon evaluates to in Mathematica on my machine.

I think this is due to the fact that one has to deal with negative and positive values so you get the 2 times larger $MachineEpsilon As to your second question: MachinePrecision is basically double precision floating point representation which means each number gets 8 bytes. • $MachineEpsilon is 2^(-n+1), where n is the number of binary bits used in the internal representation of machine-precision floating-point numbers (see $MachineEpsilon documentation). $MachinePrecision` is the number of decimal digits of precision used for machine-precision numbers (see \$MachinePrecision documentation). The two numbers are related as you noticed, but there is no inconsistency. Sep 9, 2016 at 14:44