4
$\begingroup$

I import large amounts of data, and for whatever reason the data in cases add large numbers of either '0s' or '9s'. However, as the numbers have very different relevant digits, I cannot simply use NumberForm to define the number of digits. One example of the problem is shown below, but it could also be, e.g., 102.0000000000003 or 56.99999999998. I could come up with some complicated rules, but I hope for a simpler solution.

The data are running through a script for some other corrections, in this step, I'd like to also correct these nonsense digits.

enter image description here

One example where it still does not work:

enter image description here

$\endgroup$
10
  • $\begingroup$ Can you explain how you are importing this data, particularly the actual code you're using and the format of the data? This is a floating point issue. $\endgroup$
    – Carl Lange
    Mar 18, 2020 at 10:04
  • $\begingroup$ The data are copied from a website to an Excel file, from which this is then imported. $\endgroup$ Mar 18, 2020 at 10:17
  • 3
    $\begingroup$ Can you explain why you need to change these numbers? For computaiton, it should not be needed. Is it for display only? Why does NumberForm not work for you? Can you give specific examples where NumberForm does not give you the desired display? $\endgroup$
    – Szabolcs
    Mar 18, 2020 at 12:03
  • 1
    $\begingroup$ Further to what @Szabolcs noted, some numbers have short exact representations in base 10 but not in binary. A consequence for finite binary representations is that they may not perfectly "round-trip"; output of the closest base 10 value of comparable precision might be slightly different from input. By the way, this will not happen with values that are decimal renditions of integers, such as 102.0 or 57.000. $\endgroup$ Mar 18, 2020 at 13:45
  • 1
    $\begingroup$ The reason is what Daniel said. You know that 1/3 cannot be written in full in decimal: it would require an infinite number of digits. 0.3333 is not 1/3. The same is true for 1/5 = 0.2 in binary. You have numbers like 0.0671 in your list. Those cannot be represented exactly. It's going to have to be either 0.06710000000000001 or 0.06709999999999999. The program you are importing from seems to have chosen the latter for you. $\endgroup$
    – Szabolcs
    Mar 18, 2020 at 14:19

1 Answer 1

3
$\begingroup$

I do not know the internals of Mathematica but I noticed that ToString removes digits after a second group of zeroes. Try this code.

    inp = {0.00008200000000000003`, 0.333333333333333312`, 
    0.2333999999999999976`};
    ToExpression[RowBox[{ToString[#]}]] & /@ inp // FullForm

Previous answer

The function N@Rationalize[#, 10^(-$MachinePrecision + 5)] & seems to do the trick.

    inp = {0.006000000000000538`, 0.0025800000002995675`, 
   0.19899999999989948`, 1/3} ;
    N@Rationalize[#, 10^(-$MachinePrecision + 5)] & /@ inp // FullForm
    (*List[0.006`,0.00258`,0.199`,0.3333333333333333`]*)

$MachinePrecision is basically the number of digits that your machine uses internally. The exponent -$MachinePrecision + 5 says that the denominator of Rationalize should have that number of digits.

$\endgroup$
8
  • 3
    $\begingroup$ Actually, Chop did not do anything here. It either replaces a number by zero, or it leaves it alone. What you see here is the lower default display precision in Output cells than in Input ones. $\endgroup$
    – Szabolcs
    Mar 18, 2020 at 12:01
  • $\begingroup$ Szabolcs is correct, as soon as you look at the output in full it is the same as before. I also tried Chop. $\endgroup$ Mar 18, 2020 at 14:02
  • $\begingroup$ @MockupDungeon, the answer with Chop was incorrect. Have you tried the one with Rationalize? $\endgroup$ Mar 18, 2020 at 14:42
  • $\begingroup$ Sorry, I overlooked it. I have to test it a little more, but it looks good so far! $\endgroup$ Mar 18, 2020 at 14:57
  • $\begingroup$ OK, I think this seems to work. I had to make a MachinePrecision of +6 for the large numbers and -7 for the small numbers. I couldn't fully understand how MachinePrecision works, but build in a test in case the new and old value are significantly different. I hope, I don't input some new artifacts ... $\endgroup$ Mar 18, 2020 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.