# Precision in calculation [duplicate]

I am writing some code to demonstrate the effect of the number of significant figures a coefficient has on the solution of a nonlinear equation. I define the coefficient as follows:

sx = N[1/Sqrt[3], 5]


However, when I perform the calculations I notice that changing the precision (the number 5 in the above example) does not effect the results. Why can this be the case or am I using the wrong Mathematica function? You can also try the following for a minimal working example showing that indeed the roots do not change.

nx = 1.5
ny = 1.6
nz = 1.7
sx = N[1/Sqrt[3], 1]
sy = N[1/Sqrt[3], 1]
sz = N[1/Sqrt[3], 1]

NSolve[
1/n^2 == sx^2 / (n^2 - nx^2) + sy^2 / (n^2 - ny^2) +
sz^2 / (n^2 - nz^2), n]

• nx, ny, and nz are undefined – Jason B. Apr 6 '16 at 13:33
• @JasonB I have defined them sorry for the inconvenience. – Vesnog Apr 6 '16 at 13:45
• I think that the function f in Mr. Wizard's answer linked above may be what you want. But let me ask you, when you say you want N[1/Sqrt[3], 1] which is 0.6, do you want that precision maintained throughout calculations? That is, if I enter 1/.6 it returns 1.66667 with a repeating decimal representation. But if I force the precision to be 1 on this calculation like 1/N[1/Sqrt[3], 1], then it would be rounded to 2 – Jason B. Apr 6 '16 at 14:33
• @JasonB I want that number to be regarded as 0.6 throughout the calculation. – Vesnog Apr 6 '16 at 15:02
• Then you need to round it, as I do below, – Jason B. Apr 6 '16 at 15:03

I think that the function f in Mr. Wizard's answer linked above may be what you want.

But let me ask you, when you say you want N[1/Sqrt[3], 1] which is 0.6, do you want that precision maintained throughout calculations?

That is, if I enter 1/.6 it returns 1.66667 with a repeating decimal representation. But if I force the precision to be 1 on this calculation like

1/N[1/Sqrt[3], 1]
(* 2. *)


then it would be rounded to 2. Here is the tricky part, when you multiply numbers with different precision, then the set precision is thrown out,

1.0/N[1/Sqrt[3], 1]
(* 1.73205 *)


So you may think the answer would be to wrap nx, ny, and nz in the same precision, but that seems to have no effect on the outcome of NSolve. Giving NSolve the option WorkingPrecision->1 has no effect either.

The only method that seems to influence the results from NSolve is strictly rounding the number.

Using a modified version of Mr. Wizard's function (the original doesn't have an effect here),

nRound[x_, n_] :=
Round[x, 10.^(1 - n + ⌊Log10@Abs@x⌋)];


Then we get

nx = 1.5;
ny = 1.6;
nz = 1.7;
sx = nRound[1/Sqrt[3], 1];
sy = nRound[1/Sqrt[3], 1];
sz = nRound[1/Sqrt[3], 1];

NSolve[1/n^2 ==
sx^2/(n^2 - nx^2) + sy^2/(n^2 - ny^2) + sz^2/(n^2 - nz^2), n,
WorkingPrecision -> 2]
(* {{n -> 0. + 5.66204 I}, {n ->
0. - 5.66204 I}, {n -> -1.65497}, {n -> 1.65497}, {n ->
1.5394}, {n -> -1.5394}} *)


Changing the 1 to a 3 gives

{{n -> -45.9999}, {n -> 45.9999}, {n ->
1.65466}, {n -> -1.65466}, {n -> -1.53909}, {n -> 1.53909}}

• Okay this works, but is there a one line alternative that is already built-in? I think that the function N is only for output formatting purposes. This seems a bit overkill for such a simple task. – Vesnog Apr 6 '16 at 16:06