0
$\begingroup$ 
r[i] = SetPrecision[
   Import[epath, {"Data", epath2, 9 + counts + 5*i - 2, excelfile2}], 
   prec];

epath is file name, epath2 is tab, count is row, excelfile2 is column. In the excel the cell holds the equation =.769/39.37. According to excel it .0195326390652781000000000. However in mathematica the result comes back as

r[1] = 0.01953263906527813106861480

Where is the extra 13106861480 coming from?

Whats more is if you put the equation into mathematica as .769/39.37 the result you get is 0.01953263906527813453806175 which has 3453806175. Where are these differences coming from? How come they don't change, I though it was computer noise but it's not changing. Can someone help?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ What is prec? $\endgroup$
    – Rohit Namjoshi
    Oct 17 '20 at 19:09
  • 1
    $\begingroup$ precision of argument set, 200 $\endgroup$
    – Rookey
    Oct 17 '20 at 19:18
  • $\begingroup$ When SetPrecision is used to raise the precision of a number, it augments with binary (as opposed to decimal) zeros. In[163]:= n1 = .769/39.37; n2 = SetPrecision[n1, 30] Out[164]= 0.0195326390652781345380617494811 In[165]:= RealDigits[n2, 2] Out[165]= {{1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, -5} $\endgroup$
    – Daniel Lichtblau
    Oct 18 '20 at 15:12
  • $\begingroup$ I don't understand? it just generates extra numbers? How are they not random then? and which one is accurate/correct? $\endgroup$
    – Rookey
    Oct 18 '20 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.