For questions on the use of machine-precision real numbers (also known as floats), the numbers that can be directly manipulated through the underlying numerical capabilities of your computer system.

learn more… | top users | synonyms

10
votes
3answers
589 views

Mathematica Plot: Inconsistency when plotting large values

I am working with a function in Mathematica and I am getting some inconsistencies when I plot it. As I really need to understand were this comes from I would appreciate any help. I am working with a ...
5
votes
1answer
580 views

Infinite expression 1/0. encountered - caused by precision?

Planck's law dependent on frequency rho is as follows: B[T_, h_, rho_, k_, c_] := (2 h rho^3)/c^2 1/(Exp[h rho/(T k)] - 1) As you can see, the denominator can ...
7
votes
2answers
425 views
17
votes
1answer
661 views

Quadruple-precision (128 bit) arithmetic

Is it possible to force Mathematica to perform computations with hardware supported quadruple-precision? The following test suggest, that all of the computations with fixed precision, different than ...
7
votes
1answer
130 views

Using Differences on data: trouble with floats and doubles

Consider the following data set (after I have run FullForm), which is imported from a file (stored typically as 10.040): ...
12
votes
4answers
1k views

Numerical underflow for a scaled error function

I calculate scaled error function defined as f[x_] := Erfc[x]*Exp[x^2] but it can not calculate f[30000.]. ...
5
votes
4answers
989 views

Mathematica Precision

How can I set the output precision of the following statement to 10 decimal places? I was looking through the documentation, and for some reason, all I could find was ...
7
votes
2answers
326 views

Dealing with numbers too large for machine precision in Graphics

Graphics only supports machine precision numbers (i.e. number that can be converted to machine precision). Take for example ...
26
votes
2answers
1k views

Meaning of backtick in floating-point literal

If I compute, say, 1/3//N, Mathematica displays 0.333333 as the result. When I copy that output to use elsewhere, the paste ...