4
$\begingroup$

With MachinePrecision this construction works as expected

x = N[Round[0.01499, 10^-2], MachinePrecision];
NumberForm[N[x, 1], 1, ExponentFunction -> (Null &)]

0.01

but with higher precision NumberForm fails.

x = N[Round[0.01499, 10^-2], 16];
NumberForm[N[x, 1], 1, ExponentFunction -> (Null &)]

0.*10^-2

The higher precision would be required in a case like this

d = 2;
x = N[Round[43353453524535.01499, 10^-d], 16];
p = Max[Floor[Log[10, Abs[x]]] + 1 + d, 1]
NumberForm[N[x, p], p, ExponentFunction -> (Null &)]

43353453524535.01

Any insights on how to fix case 2?

$\endgroup$
3
$\begingroup$

Updated with more details

When NumberForm acts on an extended precision number, e.g., NumberForm[number, prec], the number of digits to be shown depends on both the precision of number and the NumberForm goal prec.That is, if the precision of the number is x, and the goal is y, than the precision to be shown will be smaller of x and y. Here is an example:

NumberForm[.16`1, 2]

.2

Even though two digits of precision were requested, only one digit of precision is given in the output. Obviously, when the precision of the number is an integer, say 1, it lies on the edge of being able to show 0 or 1 digits. For some reason, NumberForm decides that a precision of 1 is not quite sufficient to show even 1 digits, and so you get:

NumberForm[.01`1, 1]

0.*10^-2

If you increase the precision of the number by just a little bit, then NumberForm is able to determine that one digit of precision can be shown:

NumberForm[.01`1.0000001, 1]

.01

So, I would add a bit of slop to the precision of the number so that it is far enough away from an integer.

Addressing the comment. Note that you must not use a non-integer in the second argument of NumberForm, because it generates an error message, and is then ignored:

NumberForm[102.1`5, 1.1]

NumberForm::iprf: Formatting specification 1.1 should be a positive integer or a pair of positive integers.

102.10

$\endgroup$
  • $\begingroup$ It does kind of work, with a warning message. e.g. d = 2; Print@FullForm[x = N[Round[7867843353453524535.01499, 10^-d], 50]]; Print[p = Max[Floor[Log[10, Abs[x]]] + 1 + d, 1] + 0.0000001]; Quiet[NumberForm[N[x, p], p, ExponentFunction -> (Null &)]] However, it's a bit of a fudge. $\endgroup$ – Chris Degnen Oct 30 '18 at 16:30
  • $\begingroup$ @ChrisDegnen See update. It's a shame that NumberForm and the FE don't agree on the number of digits to display, but the workaround is simple. $\endgroup$ – Carl Woll Oct 30 '18 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.