7
$\begingroup$
$MachineEpsilon
x = 0.0000000000000001
% + 1
% - 1

As expected the final result equals zero, since x < MachineEpsilon ($2.22045\times10^{-16}$).

However, if I double x (to 0.0000000000000002), then the final result should in my understanding still be zero, since x is still smaller than the MachineEpsilon, but... to my surprise the final result isn't zero! Question: why is this?

Note 1: If I triple x (to 0.0000000000000003), then x is larger than the MachineEpsilon and indeed the final result is bigger than zero.

Note 2: On my computer the value for MachineEpsilon is $2.22045\times10^{-16}$ Is this the same as on any computer using Mathematica by the way?


EDIT: I noticed that in binary form:

  1. The MachineEpsilon is $1.\times2^{-52}$
  2. And 0.0000000000000002 is $1.1100110100101011001\times2^{-53}$

Code:

BaseForm[$MachineEpsilon, 2]
x = BaseForm[0.0000000000000002, 2]
% + 1
BaseForm[% - 1, 2]

Therefore, could it be that $1.1100110100101011001\times2^{-53}$ is rounded up to $1.\times2^{-52}$ if $1$ is added? Which is precisely equal to the MachineEpsilon and that therefore the final result above with 0.0000000000000002 isn't zero but... the MachineEpsilon!?

$\endgroup$
  • 2
    $\begingroup$ In the past $MachineEpsilon in Windows-based and OS-X-based systems differed because the Windows-based systems used Intel processors and Apple used Motorola/IBM processors. Apple changed over to Intel in 2006. So now it would be hard to find a system running Mathematica that doesn't give 2.220446049250313`*^-16 for $MachineEpsilon because that is the value retuned by any system running an Intel Core i3, i5, or i7 processor(s). $\endgroup$ – m_goldberg Sep 9 '16 at 14:28
  • $\begingroup$ Log[2, $MachineEpsilon] evaluates to -52. and RootApproximant[$MachineEpsilon] === 2^-52 evaluates to True $\endgroup$ – Bob Hanlon Sep 9 '16 at 15:19
4
$\begingroup$

I suppose this is due to rounding. For as long as the value is smaller than half of $MachineEpsilon it yields 0, else it rounds up to $MachineEpsilon.

Given x*i+1-1

For, i<0.5*10^15 $MachineEpsilon I get zero,
while for i>0.5*10^15 $MachineEpsilon i get $MachineEpsilon.

And regarding your second question: On my laptop

$MachineEpsilon = 2.22045×10−16

It might be a naive way to show it, but here's an illustration.

DiscretePlot[{(x*i) + 1 - 1}, {i, 1.1101, 1.11035, 0.000001}]

enter image description here

$\endgroup$
3
$\begingroup$

On a binary machine with $n$-bit floating point numbers, machine $\epsilon$ is the smallest number such that $1 + \epsilon$ can be represented in $n$ bits. This is $$1.000\cdots01$$ where the length of $1$s and $0$s is $n$; in other words $$1 + \epsilon = 2^0+2^{-(n-1)}\,.$$ For the standard IEEE binary64 floating point number $n = 53$ and $$\epsilon = 2^{-52}\,.$$

So that's the story for binary machines. When you start to investigate machine $\epsilon$ on such a machine using decimal numbers, you have to consider rounding error and how arithmetic operations are carried out. Consider 1 * 10^-16 and 2 * 10^-16 in binary:

BaseForm[0.0000000000000001, 2]
BaseForm[0.0000000000000002, 2]
(*
  1.1100110100101011001 * 2^-54
  1.1100110100101011001 * 2^-53
*)

Now an arithmetic operation is carried out by the CPU with extra guard bits, so the the result is equal to the exact result of the operation rounded to $n$ bits. So $$\eqalign{ 1. &+\ 2. * 10^{-16} = \cr &1.00000000000000000000000000000000000000000000000000001110011\cdots \cr }$$ rounded to 53 bits on a binary64 machine, which yields, $$1.0000000000000000000000000000000000000000000000000001$$ If we compare them side by side,

1.00000000000000000000000000000000000000000000000000001110011...
1.0000000000000000000000000000000000000000000000000001

we see that the 54th & 55th bits cause the 53rd bit to be rounded up to 1, just as the OP suggested. Hence

x = 0.0000000000000002
% + 1
% - 1

has a nonzero final result, namely 2.^-52. On the other hand, in $1. + 1. * 10^{-16}$, there is another 0 between the first and second 1s, so that the 54th & up bits are 01.... Thus the 53rd bit is rounded to 0, and the result is 1.. Subtracting 1 from it yields zero, as in the OP's first example.

$\endgroup$
2
$\begingroup$

What happens when adding 1 to a number close to MachineEpsilon?

Some of the confusion in the question is caused by the way Mathematica displays machine-precision numbers. For example, Mathematica's "nice" default settings make it appear that there's no difference between 1.0 and 1.0 + $MachineEpsilon.

1.0
1.0 + $MachineEpsilon

The displayed results are the same, that is, 1.. However, we can use RealDigits to show that 1.0 + $MachineEpsilon is not the same as 1.0 at machine-precision (assuming MachinePrecision is $\frac{53 \log (2)}{\log (10)}$≈15.9546).

RealDigits[1.0, 2] (* binary digits *)
RealDigits[1.0 + $MachineEpsilon, 2]

The results are:

{{1, 0, <<50 zeros>>, 0}, 1} (* least-significant digit is 0 *)
{{1, 0, <<50 zeros>>, 1}, 1} (* least-significant digit is 1 *)

The least-significant binary digit shows that 1.0 and 1.0 + $MachineEpsilon are in fact different, and demonstrates that MachineEpsilon is "the minimum positive machine-precision number which can be added to 1.0 to give a result distinguishable from 1.0."

The point is that we can't rely on Mathematica's "nice" display of machine-precision numbers. We need to look as closely as possible at the binary result.

Case #1, where x = 0.0000000000000001

Consider the result of adding 1 to 0.0000000000000001.

x = 0.0000000000000001;
RealDigits[x + 1, 2]}

The result is {{1, 0, <<50 zeroes>> 0}, 1}, and notice the least-significant digit is 0.

Case #2, where x = 0.0000000000000002

x = 0.0000000000000002;
RealDigits[x + 1, 2]

This time, the result is {{1, 0, <<50 zeros>> 1}, 1}. Compare to case #1, where the least-significant digit was 0. This shows how decimal-to-binary conversion and using a number twice as large as case #1 produces a different result, and that the result is MachineEpsilon larger.

The different least-significant binary digits are the reason why the case #2 result isn't zero, and it is in fact the same as MachineEpsilon.

The case where x = 0.0000000000000003 is the same as x = 0.0000000000000002.

x = 0.0000000000000003;
RealDigits[x + 1, 2]

Another way to look at these results is to use extended-precision numbers instead of machine-precision values. For example, using case #2:

x = 0.0000000000000002;
SetPrecision[x + 1, 22]

The result is 1.000000000000000222045. Notice the the digits 222045 from the value of MachineEpsilon.

To summarize, Mathematica displays machine-precision numbers in a "nice" way that doesn't show the full precision of the result of a calculation. We need to use RealDigits or use extended precision to see the results of adding 1 to numbers close to MachineEpsilon.

Is MachineEpsilon the same as on any computer using Mathematica?

The value of MachineEpsilon is determined by the internal representation of machine-precision floating-point numbers. The value of MachinePrecision is $\frac{53 \log (2)}{\log (10)}$≈15.9546, when a computer uses IEEE Standard 754-1985 double-precision, 64-bit floating-point numbers. The value 53 comes from the number of bits used to store the mantissa (52) plus an assumed, normalized 1 bit to the left of the binary mantissa.

$\endgroup$
0
$\begingroup$

Seems like $MachinePrecision and $MachineEpsilon are inconsistent with each other.

Logically MachineEpsilon should be equal to $\frac{1}{10^{$MachinePrecision}}$ but in fact MachineEpsilon = $ \frac{2}{10^{$MachinePrecision}} $

enter image description here

So in fact when you multiply your number by 2 you get a number that is larger than the smallest allowed number by MachinePrecision.

Also the threshold that @Dede found correspond to $MachineEpsilon/2 Which in fact the (effective $MachineEpsilon) = $\frac{1}{10^{$MachinePrecision}}$ and not whatever $MachineEpsilon evaluates to in Mathematica on my machine.

I think this is due to the fact that one has to deal with negative and positive values so you get the 2 times larger $MachineEpsilon

As to your second question: MachinePrecision is basically double precision floating point representation which means each number gets 8 bytes.

$\endgroup$
  • 2
    $\begingroup$ $MachineEpsilon is 2^(-n+1), where n is the number of binary bits used in the internal representation of machine-precision floating-point numbers (see $MachineEpsilon documentation). $MachinePrecision is the number of decimal digits of precision used for machine-precision numbers (see $MachinePrecision documentation). The two numbers are related as you noticed, but there is no inconsistency. $\endgroup$ – creidhne Sep 9 '16 at 14:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.