# Questions tagged [precision-and-accuracy]

The tag has no usage guidance.

527 questions
Filter by
Sorted by
Tagged with
40 views

### Rationalizing only specific terms in an expression

How can I change an expression like A = (-1. a)/(x + y) - (2.1 b)/(x + y) + 4.1 + 2. x + (3. r s)/(x + y) so that -1. a becomes ...
45 views

### Problem with the precision while plotting a figure

I am trying to plot eigenvalues of a 16x16 matrix. Below you can see the code. I get some errors like “precision may be lost”, and “Eigenvalues at some positions are expected to be the same”. As a ...
62 views

### Accuracy of Mod[x, 1] and FractionalPart[x]

The difference between exact and numerical computation is clear to me. But why is ...
230 views

### Computing separatrix orbit

I would like to numerically compute the separatrix orbit for a particle in a few potential wells, given by the negative of ...
24 views

### How to avoid machine underflow for plotting and integrating?

I am trying to integrate the following integrand, which I have plotted: ...
68 views

### Lowest Eigenvalue of a large sparse matrix without choking up the RAM (Corrected) [closed]

I am trying to find the lowest eigenvalue of a large sparse matrix of dimension N (=$3^n$, where n is number of particles in system ) using ...
193 views

### NIntegrate::ncvb: NIntegrate failed to converge to prescribed accuracy

The integration is: ...
36 views

### Problem with label and precision

I need to define a function that depends on a variable taking values on discrete values, for example: ...
145 views

104 views

119 views

### Does SetPrecision[x, Infinity] expose the internal exact number in the approximated number?

What I already know (maybe) : My theory about Mathematica's way of implementing approximated number An number approX with arbitrary precision ...
36 views

### Can I configure a guaranteed precision for NIntegrate on a monotone function?

I know there are some great posts already about why PrecisionGoal->n doesn't guarantee the result of NIntegrate will actually ...
44 views

### Help with NMinimize and setting limits

Im trying to minimize the error of a function Penality that i have defined and am using the following code. ...
106 views

### Why does precision decrease after increasing PrecisionGoal and WorkingPrecision on this integral?

I am trying to calculate a messy double-integral to many digits of precision. As shown below, the precision seems to increase reliably as WorkingPrecision goes from ...
104 views

### How to improve working precision?

I'm trying to fit the array data as follow: NonlinearModelFit[ data, s*t^r + (A/(w*(Pi/2)^0.5))*Exp[-2*((t - u)/w)^2], {s, r, A, w, u}, t]; But when ...
65 views

### Numerical evaluation expression not returning result in requested precision

I have a long list of descriptive statistical quantities for a univariate list of data. The problem is that the set number of significant figures is ignored by the ...
85 views

### ListLinePlot works for data similar up to 9 decimal numbers but does not work when similarity is up to 11 decimal numbers. Why?

Through the following code, Tcvalues are generated: ...
98 views

### A pattern replacement to force Mathematica to replace inexact 0 multiplication by zero

Hi Consider the following expression which contains multiplication by inexact 0 ...
32 views

### Setting the tolerance for Equal[] [duplicate]

I would like to control how Equal[] works and allow a certain error associated with it. I would like numerical values which are, say, within one-thousandth of one ...