# N[…, 5] does not returns what I expected

I have this

Xt =
{{1., 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{1, 1.5, 1.5,1.5, 2, 2, 2.2, 2.4, 2.5, 2.5, 2.8, 2.8, 3, 3, 3.2, 3.3}};
Yt =
{101.4, 117.4, 117.1, 106.2, 131.9, 146.9, 146.8, 133.9, 111.3, 123, 125.1, 145.2, 134.3, 144.5, 143.7, 146.9};

N[Inverse[Xt.Transpose[Xt]].Xt.Yt, 5]


and the last evaluation returns the same result as if there was no N[..,..]; i.e.,

{93.3422, 15.6485}

Why and how do I get output to five decimals?

• Welcome to Mathematica SE! Generally we try to only use the "bugs" tag once other users have verified that a function is not working the way the documentation suggests it should. – MassDefect Feb 9 '19 at 7:31
• well no one told me about it.. I'll edit :). – America Feb 9 '19 at 7:34
• It's no problem! I just thought I'd leave a comment for future reference. I had no idea that's how it worked when I first started here either! – MassDefect Feb 9 '19 at 7:48
• One thing every Mathematica user must learn is that the 2nd argument of N has no effect on machine precision numbers, which are the kind of numbers you are using. – m_goldberg Feb 9 '19 at 9:07
• I think you might find reading this answer helps your understanding of N, which is a more sophisticated function than most beginners think it is. – m_goldberg Feb 9 '19 at 9:21

Here is one solution.

Xt = Rationalize@{{1., 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1}, {1, 1.5, 1.5, 1.5, 2, 2, 2.2, 2.4, 2.5, 2.5, 2.8, 2.8, 3, 3,
3.2, 3.3}};
Yt = Rationalize@{101.4, 117.4, 117.1, 106.2, 131.9, 146.9, 146.8,
133.9, 111.3, 123, 125.1, 145.2, 134.3, 144.5, 143.7, 146.9};
Inverse[Xt.Transpose[Xt]].Xt.Yt


$$\left\{\frac{892351}{9560},\frac{3740}{239}\right\}$$

N[Inverse[Xt.Transpose[Xt]].Xt.Yt, 5]


{93.342, 15.649}

N[Inverse[Xt.Transpose[Xt]].Xt.Yt, 10]


{93.34215481, 15.64853556}

The documentation for N says that N[expr, n] attempts to give a result with n-digit precision. In the Wolfram Language, or any language, precision is not the same as decimal places. While it might seem like N is there to give us results with a certain number of decimal places, it actually has more to do with the amount of precision used in calculations behind the scenes. It can be especially confusing as sometimes N does close to what we expect as in N[Pi, 10] which gives us $$\pi$$ with 10 digits (9 decimal places).

There is a different function called NumberForm that is designed to let us have more control over the display of numbers.

We can use:

NumberForm[Inverse[Xt.Transpose[Xt].Xt.Yt, {20, 5}]


yields

{93.34215, 15.64854}

The first number in NumberForm (i.e. 20) is the total number of digits. We can set this to Infinity and it won't make a difference, but it shouldn't be less than the total number of digits you want to display. The second number (i.e. 5) is the number of decimal places.

According to this tutorial:

MachinePrecision is considered less precise than any other precision. This is because, while machine-precision numbers always store slightly under 16 digits, no specific number of those digits are known to be correct.

We're working with MachinePrecision, i.e.

Precision[Inverse[Xt.Transpose[Xt]].Xt.Yt]

MachinePrecision


and so N[Inverse[Xt.Transpose[Xt]].Xt.Yt, 5] needs to somehow upsample the precision of the result and that is an ill-defined operation.

In fact the ref page for N states

Unless numbers in expr are exact, or of sufficiently high precision, N[expr, n] may not be able to give results with n-digit precision.

To see why this is ill-defined, consider the reverse. Let's go from higher precision to lower precision.

x1 = N[Pi, 10]


$$3.1415926{\color{red}5}4$$

x2 = N[Pi + 10^-8, 10]


$$3.1415926{\color{red}6}4$$

N[{x1, x2}, 5]


{3.1416, 3.1416}

So we see if we reverse this example, going from precision 5 to precision 10 is ambiguous in the choices of the 6th - 10th digits.

If you'd like to upsample in a consistent way, use SetPrecision:

When SetPrecision is used to increase the precision of a number, the number is padded with zeros. The zeros are taken to be in base 2. In base 10, the additional digits are usually not zeros.

SetPrecision[Inverse[Xt.Transpose[Xt]].Xt.Yt, 5]

{93.342, 15.649}


If you only care about the display of the answer, consider using NumberForm:

NumberForm[Inverse[Xt.Transpose[Xt]].Xt.Yt, 5]

{93.342, 15.649}