Questions tagged [probability-or-statistics]

Questions about systematic data collection and organization, or the application of probability theory to model the inherent patterns and properties of sampled data, underlying data distribution(s) or random processes.

Filter by
Sorted by
Tagged with
48
votes
5answers
13k views

Simultaneously fitting multiple datasets

What is the proposed approach if one wants to simultaneously fit multiple functions to multiple datasets with shared parameters? As an example consider the following case: We have to measurements of ...
44
votes
7answers
7k views

Efficient way to generate random points with a predefined lower bound on their pairwise Euclidean distance

Using Mathematica what is an efficient way to generate a list of $n$ random two dimensional points $\{x_i,y_i\}$ where $i=1,...,n$ so that no two points $p_1$ and $p_2$ in the list has an Euclidean ...
35
votes
6answers
16k views

Estimate error on slope of linear regression given data with associated uncertainty

Given a set of data, is it possible to create a linear regression which has a slope error that takes into account the uncertainty of the data? This is for a high school class, and so the normal ...
40
votes
5answers
6k views

Efficiently generating n-D Gaussian random fields

I am interested in an efficient code to generate an $n$-D Gaussian random field (sometimes called processes in other fields of research), which has applications in cosmology. Attempt I wrote the ...
47
votes
3answers
5k views

RandomVariate from 2-dimensional probability distribution

A probability distribution can be created in Mathematica (I am using 8.0.1) with e.g. ...
24
votes
3answers
12k views

How to generate a RandomVariate of a custom distribution?

I'm trying to generate a pseudorandom variate out of a custom distribution. Suppose I want define a custom distribution, and for the sake of simplicity I define a Poisson distribution (the ...
34
votes
4answers
2k views

Which Distributions can be Compiled using RandomVariate

Recently, Oleksandr kindly showed a list of Mathematica commands that can be compiled. RandomVariate was part of that list. However, whether this can be compiled depends upon the distribution that is ...
16
votes
2answers
10k views

Performing a chi-square goodness of fit test

How would I perform a chi-square goodness of fit test? I have tried the following, where my data consists of the observed values whilst the data pair contains the correct values. Then I use the ...
29
votes
3answers
4k views

Standard errors for maximum likelihood estimates in FindDistributionParameters

I am trying to fit a non-standard PDF to data and FindDistributionParameters works great, and gives me the parameters of the distribution back using maximum ...
9
votes
3answers
3k views

Ηow to create an interpolated CDF from its samples?

I want to use a distribution I have only aggregate statistics on, namely its CDF sampled at certain points. I would like to keep it "nonparametric" (remain noncommittal on the parametric form), but I ...
2
votes
3answers
340 views

Evaluate a certain three-dimensional constrained integral

The result of the three-dimensional integration ...
10
votes
1answer
1k views

How can we create a histogram distribution from binning data?

HistogramDistribution takes a list of data points, creates a histogram from them, and returns this histogram in a form that can be used as a distribution. I ...
31
votes
3answers
4k views

Histograms with pre-counted data

I've got some large data sets which have been counted but not binned already - essentially, a list of pairs of values (not bins) and counts.* (Or, equivalently, it's been binned into too-small bins.) ...
37
votes
1answer
902 views

How to augment the realm of functions Mathematica thinks it knows how to integrate symbolically

My question involves extending the functionality of Integrate over specific integrals in the most generic manner. Specifically, is it possible to "hack into" ...
17
votes
3answers
5k views

How to remove outliers from data

I have a data with noise which some times includes significant outliers. The position of the outliers are random. For example: ...
11
votes
2answers
8k views

How to draw confidence ellipse from a covariance matrix?

I am studying a two-dimensional dataset, whose mean vector and covariance matrix are the following: ...
16
votes
3answers
3k views

How can I calculate the correlation dimension and/or the Lyapunov exponent of a time series using Mathematica

I have a pretty long time series (4000 data points). How can I calculate the Correlation dimension and/or Lyapunov exponent for such data? Please also advise on how to import the data as well.
32
votes
6answers
3k views

Generate two random numbers with constraints

I want to generate two random numbers, $p$ and $q$, between $0.5$ and $1$. They are connected by the constraint $1/(2q) > p$. How do I generate $p$ and $q$?
8
votes
3answers
2k views

Contour lines over SmoothDensityHistogram

I am using SmoothDensityHistogram on a data set of the form {{x1, y1}, {x2, y2}, …, {x_n, y_n}}, and I would like to also show ...
3
votes
2answers
624 views

Defining symbolic expectation function

I want to define the standard expectation $E$ operator in Mathematica. In particular, I want it to satisfy, $E[ c + a \cdot X_1^{n_1} X_2^{n_2} X_3^{n_3} X_4^{n_4} + b \cdot Y_1^{m_1} Y_2^{m_2} Y_3^...
8
votes
3answers
12k views

How to use a 3×3 covariance matrix to plot an error ellipsoid?

I have a 3×3 error covariance in Mathematica, but I don't know how to use it for plotting the error ellipsoid. It would be great if you can show me how I can do that for the below covariance matrix: <...
16
votes
2answers
4k views

Finding distribution parameters of a gaussian mixture distribution

Short version: how to estimate the parameters of a mixture of multivariate normal distributions (i.e.: Gaussian mixture model)? Long version. I am trying to estimate the parameters of a mixture of ...
8
votes
1answer
858 views

Problems encoding a Bayesian Network with just five nodes using ProbabilityDistribution

Question summary I had recently asked this question where problems encoding a Bayesian Network were linked to the use of MultinomialDistribution. While that ...
13
votes
2answers
2k views

Filtering and Replacing outliers

I have a set of data that measurements of temperature vs time (date). It is easy to realize when the sensor went wrong as one might see a blip on the data. How to teach to Mathematica to: 1) Detect ...
7
votes
3answers
1k views

Removing background from peak function

Let's say I have a a very large list of data in the format {{n,f[n]}} so something like {{1,600},{2,700},{3,1100}...} and if this data is plotted, then it will look like a sum of Gaussian functions+a ...
6
votes
0answers
212 views

Does DegreeGraphDistribution sample uniformly?

Bug introduced in 10.0 or earlier and persisting through 12.1 The first symptom (histogram below) is gone in 12.0. The second (fraction of connected graphs) persists, showing that the sampling is ...
10
votes
3answers
677 views

Can I use Compile to speed up InverseCDF?

I'm repeatedly issuing the command P = InverseCDF[NormalDistribution[0, 1], T] where T is either a 36- or 64-vector of real numbers (points in the unit ...
38
votes
3answers
3k views

Computational Bayesian analysis in Mathematica: Any plans to develop MCMC?

Does anyone know if there are any plans to develop an MCMC capability in Mathematica? My reasoning for asking is that as it stands, I can't seem to find any 'out-of-the-box' functions/capabilities ...
32
votes
5answers
8k views

Showing the correlation of two variables using a plot

I have a list of $\{x_i,y_i\}$ pairs and I want to show that $x_i$'s and $y_i$'s have positive correlation. The result of ...
14
votes
2answers
1k views

How to do Independent Component Analysis?

The new experimental function in 10.3.1 DimensionReduce[], has the following three options for Method: ...
24
votes
2answers
1k views

Given an exact formula, how can Mathematica find a probability distribution whose PDF matches it?

So, given some data, Mathematica 10.2 can now attempt to figure out what probability distribution might have produced it. Cool! But suppose that, instead of having data, we have something that is in ...
13
votes
3answers
958 views

Different calculation of median and quartiles, bug in BoxWhiskerChart?

When you plot a dataset with BoxWhiskerChart passing over the box in the plot with your mouse pointer shows some specifics about the dataset (min and max values, 1/...
13
votes
1answer
2k views

How does Mma compute Confidence Intervals?

I am trying to understand how Mma computes the Confidence Intervals after a NonlinearModelFit. Consider the following example: ...
14
votes
3answers
2k views

How can I tell mathematica to generate an histogram from nominal data?

If I write: Histogram[{1, 1, 1, 2, 2, 2}] I get a nice histogram chart but if I write ...
10
votes
1answer
706 views

Representative Smooth Kernel Distribution from Truncated Distribution

I am trying to produce a better distribution from a dataset that is bounded to be greater than 0. Here is an example distribution from the documentation that mimics the behavior of the actual dataset: ...
6
votes
1answer
281 views

Determining the dimension of a probability distribution

Functions like Mean and RandomVariate clearly infer the dimension of the distribution passed to them. One can also usually ...
6
votes
2answers
2k views

Empirical Cumulative Distribution Function

I have a data set of the form $d=\{(y_1,x_1),(y_2,x_2)...(y_n,x_n)\}$ for a large $n$. A non-parametric plot of this data (a scatter plot where all observations are sorted along $x$ and joined with a ...
5
votes
3answers
923 views

Plotting the convolution of the same probability density function with itself

I would like to compute the integral $$q_n(x)=\int_{B}^A q_{n-1} (\omega)f(x-\omega)\mathrm{d} \omega,\quad q_1=f,\quad n\geq 1.$$ for a given density function $f$. Basically it is the convolution ...
4
votes
4answers
462 views

How to partition a list into sublists in a similar way to Histogram

I have a list of list as below (I show part of it) ...
7
votes
1answer
153 views

Integrate yielding a ConditionalExpression but I don't think the condition is necessary

Suppose I take the PDF of the LogNormal distribution with parameters m and s evaluated at x. I obviously get an expression involving m. I now want to integrate that expression not with respect to x ...
22
votes
1answer
1k views

Backtesting a Probability of Default (PD) model

Background PD models Financial institutions use Probability of Default (PD) models for various purposes such as client acceptance, provisioning and regulatory capital calculation as required by the ...
19
votes
2answers
3k views

Obtaining joint distributions and conditional distributions using Mathematica

I have two multivariate Gaussian distributions $p(x)$ and $p(z)$ with mean vectors $m_x$ and $m_z$, and covariance matrices $\Sigma_x$ and $\Sigma_z$. my model is a simple linear model $x = W z+n$ ...
23
votes
2answers
579 views

Unexpected behavior of confidence bands for data presenting two regions with uneven noise

Let's generate some noisy data with uneven noise. ...
21
votes
2answers
1k views

Efficient Generation of Random Variates from a Copula Distribution

I have a 7-asset portfolio for which I want to simulate daily log-deltas using a Student T copula. The marginal distributions are all either Stable or TsallisQGaussian. Using NMaximize, I have ...
20
votes
3answers
1k views

How to efficiently find moments of a multinormal distribution?

Update: Starting from V10.0 the build-in Moment is fast enough for practical use. I have a multinormal distribution with covariance matrix $\sigma$ and zero mean. ...
19
votes
2answers
932 views

Visualization: elliptical insights into data sets…

I was recently reading the following paper on visualization techniques using ellipses to gain statistical insights. Elliptical visualization for looking at correlations has briefly been touched on ...
11
votes
2answers
1k views

How to implement Kullback-Leibler divergence using Mathematica's probability and distribution functions?

Last year Jens answered a question about how to implement Kullback-Leibler divergence in the discrete case. He gave this answer: ...
9
votes
2answers
4k views

How to set up a simple Monte Carlo simulation?

there is a 1% chance that there will be a scattering event, if there is a scattering event the direction of scattering will be determined by a random sample of the custom[a_] function and if there is ...
10
votes
1answer
842 views

BoxWhiskerChart - 10-90th percentile instead of min - max

I am trying to draw a custom BoxWhiskerChart with alternative values determining fences. I have data that has a lot of outliers and I have to compare it to another ...
7
votes
1answer
399 views

How to derive SinglePredictionBands from a NonlinearModelFit model

I have found NonlinearModelFit to be extremely useful in my work. Here I have fit 4 points to a function and am being asked to explain how the 95% SinglePredictionBands associated with the plot below ...

1
2 3 4 5