Questions tagged [probability-or-statistics]
Questions about systematic data collection and organization, or the application of probability theory to model the inherent patterns and properties of sampled data, underlying data distribution(s) or random processes.
1,872
questions
52
votes
5
answers
15k
views
Simultaneously fitting multiple datasets
What is the proposed approach if one wants to simultaneously fit multiple functions to multiple datasets with shared parameters?
As an example consider the following case: We have to measurements of ...
51
votes
3
answers
6k
views
RandomVariate from 2-dimensional probability distribution
A probability distribution can be created in Mathematica (I am using 8.0.1) with e.g.
...
46
votes
7
answers
9k
views
Efficient way to generate random points with a predefined lower bound on their pairwise Euclidean distance
Using Mathematica what is an efficient way to generate a list of $n$ random two dimensional points $\{x_i,y_i\}$ where $i=1,...,n$ so that no two points $p_1$ and $p_2$ in the list has an Euclidean ...
44
votes
1
answer
2k
views
How to fit a linear model in the Bayesian way in Mathematica?
Basically, I'm looking for the Bayesian equivalent of LinearModelFit. As of the moment of writing, Mathematica has no real (documented) built-in
functionality for ...
41
votes
3
answers
4k
views
Computational Bayesian analysis in Mathematica: Any plans to develop MCMC?
Does anyone know if there are any plans to develop an MCMC capability in Mathematica?
My reasoning for asking is that as it stands, I can't seem to find any 'out-of-the-box' functions/capabilities ...
40
votes
5
answers
8k
views
Efficiently generating n-D Gaussian random fields
I am interested in an efficient code to generate an $n$-D Gaussian random field (sometimes called processes in other fields of research), which has applications in cosmology.
Attempt
I wrote the ...
39
votes
1
answer
1k
views
How to augment the realm of functions Mathematica thinks it knows how to integrate symbolically
My question involves extending the functionality of Integrate over specific integrals in the most generic manner.
Specifically, is it possible to "hack into" ...
35
votes
6
answers
18k
views
Estimate error on slope of linear regression given data with associated uncertainty
Given a set of data, is it possible to create a linear regression which has a slope error that takes into account the uncertainty of the data?
This is for a high school class, and so the normal ...
35
votes
4
answers
2k
views
Which Distributions can be Compiled using RandomVariate
Recently, Oleksandr kindly showed a list of Mathematica commands that can be compiled.
RandomVariate was part of that list. However, whether this can be compiled depends upon the distribution that is ...
33
votes
6
answers
4k
views
Generate two random numbers with constraints
I want to generate two random numbers, $p$ and $q$, between $0.5$ and $1$.
They are connected by the constraint $1/(2q) > p$.
How do I generate $p$ and $q$?
33
votes
5
answers
10k
views
Showing the correlation of two variables using a plot
I have a list of $\{x_i,y_i\}$ pairs and I want to show that $x_i$'s and $y_i$'s have positive correlation. The result of ...
33
votes
2
answers
1k
views
Huge bug involving MultinormalDistribution?
Bug introduced in 8.0 and fixed in 11.1
I found a strange behavior regarding the CDF of the bivariate Normal distribution
...
32
votes
3
answers
4k
views
Histograms with pre-counted data
I've got some large data sets which have been counted but not binned already - essentially, a list of pairs of values (not bins) and counts.* (Or, equivalently, it's been binned into too-small bins.) ...
30
votes
4
answers
5k
views
Standard errors for maximum likelihood estimates in FindDistributionParameters
I am trying to fit a non-standard PDF to data and FindDistributionParameters works great, and gives me the parameters of the distribution back using maximum ...
29
votes
2
answers
3k
views
How to implement Markov Chain Monte Carlo with built-in functions?
These days I'm trying to conduct a model sensitivity test which is heavily based on the Markov Chain Monte Carlo simulation approach.
And I find this 'MCMC' package that can perform Markov Chain ...
26
votes
2
answers
1k
views
Given an exact formula, how can Mathematica find a probability distribution whose PDF matches it?
So, given some data, Mathematica 10.2 can now attempt to figure out what probability distribution might have produced it. Cool! But suppose that, instead of having data, we have something that is in ...
26
votes
2
answers
2k
views
Efficiently generating 2-D Gaussian random fields on the sphere
Context
A couple of years ago I posted this question for an efficient code to generate an n-D Gaussian random field (sometimes called processes in other fields of research), which has applications in ...
25
votes
0
answers
480
views
Summer Solstice Spurious Correlations Challenge [closed]
I came across this hilarious web site that has charts of bizarre "correlations" between disparate categories. Results like this:
In the spirit of my summer solstice shedding spuriously earned rep. ...
24
votes
3
answers
13k
views
How to generate a RandomVariate of a custom distribution?
I'm trying to generate a pseudorandom variate out of a custom distribution. Suppose I want define a custom distribution, and for the sake of simplicity I define a Poisson distribution (the ...
23
votes
3
answers
2k
views
How to efficiently find moments of a multinormal distribution?
Update: Starting from V10.0 the build-in Moment is fast enough for practical use.
I have a multinormal distribution with covariance matrix $\sigma$ and zero mean. ...
23
votes
2
answers
2k
views
Calculating probabilities symbolically
Is there a way to solve for statistical quantities analytically/symbolically in Mathematica?
example 1:
Lets say that I want to do a calculation that requires Bayes theorem.
I know p(a), p(b) and p(...
23
votes
2
answers
618
views
Unexpected behavior of confidence bands for data presenting two regions with uneven noise
Let's generate some noisy data with uneven noise.
...
22
votes
3
answers
2k
views
Efficient way to simulate thousands of Markov chains
I am currently trying to simulate relaxation of a protein population while maintaining the stochastic properties of the system. For this, I used a Markov chain to describe the temporal evolution of ...
22
votes
4
answers
2k
views
Are Easter days normally distributed?
How can Mathematica be used to check if Easter days are normally distributed?
By definition, Easter day is the first Sunday after the first full moon after the spring equinox (so between March 23d ...
22
votes
2
answers
1k
views
Curve fitting by running through hundreds of models and return the one with best fit
In the past, I used to solve a lot of my regression analysis (curve fitting) problems with a program called DataFit which runs on Windows. It has hundreds of regression models which it runs through in ...
22
votes
2
answers
1k
views
Efficient Generation of Random Variates from a Copula Distribution
I have a 7-asset portfolio for which I want to simulate daily log-deltas using a Student T copula. The marginal distributions are all either Stable or TsallisQGaussian. Using NMaximize, I have ...
22
votes
1
answer
3k
views
Backtesting a Probability of Default (PD) model
Background
PD models
Financial institutions use Probability of Default (PD) models for various purposes such as client acceptance, provisioning and regulatory capital calculation as required by the ...
21
votes
5
answers
1k
views
Sisyphus Random Walk
A Sisyphus Random Walk evolves as follows: With probability (r) you advance the position of the walker by +1. With probability (1-r) the walker resets at x0 = 0.
To simulate the probability of reset ...
21
votes
3
answers
731
views
More efficient method to compute moments of the Johnson $S_B$ distribution
Here is a very specific feature request. I need
Mean[JohnsonDistribution["SB", γ, δ, 0, 1]]
When I issue e.g.
...
21
votes
4
answers
519
views
Negative probability?
Bug introduced in 9.0.1 and fixed in 10.0.2
I am trying to get the sum of the squares of seven random variables, all uniformly distributed. This is what I tried.
...
20
votes
4
answers
2k
views
Using Mathematica to solve a probability problem
The problem is from Principles of Statistics by M.G. Bulmer:
In a certain survey of the work of chemical research workers, it was found, on the basis of extensive data, that on average each man ...
20
votes
3
answers
7k
views
How to remove outliers from data
I have a data with noise which some times includes significant outliers. The position of the outliers are random.
For example:
...
20
votes
2
answers
1k
views
Visualization: elliptical insights into data sets…
I was recently reading the following paper on visualization techniques using ellipses to gain statistical insights. Elliptical visualization for looking at correlations has briefly been touched on ...
19
votes
5
answers
2k
views
Add a rug representation to plot
This question came to me when I read How convert list of numbers to list of points on x-axis?
@Mr.W suggested Interlacing a single number into a long list for the question. @Artes provides a very ...
19
votes
4
answers
3k
views
Bayesian Inference with Continuous prior distribution
Assume that we have a prior distribution for the probability of success as follows.
p = BetaDistribution[6, 14]
which we can graph as this
...
19
votes
2
answers
3k
views
Obtaining joint distributions and conditional distributions using Mathematica
I have two multivariate Gaussian distributions $p(x)$ and $p(z)$ with mean vectors $m_x$ and $m_z$, and covariance matrices $\Sigma_x$ and $\Sigma_z$.
my model is a simple linear model $x = W z+n$ ...
19
votes
1
answer
290
views
HistogramList is much Slower in 10.2 vs 10.1
While I was running a code I wrote using version 10.1, I noticed that in version 10.2 the running time of the same code was more than twice as long. After some digging, I figured out the culprit is <...
19
votes
1
answer
288
views
Possible bug in OrderDistribution? [Update: Confirmed by Wolfram Research and enqueued.]
Bug Introduced in 11.3 or earlier
Initial testing indicates this has been addressed in 12.0
Observe:
...
18
votes
3
answers
2k
views
What is the average of rolling two dice and only taking the value of the higher dice roll?
I am referencing to this question on mathunderflow:
What is the average of rolling two dice and only taking the value of the higher dice roll?
Now I tried:
...
18
votes
4
answers
702
views
Speeding up Gaussian sampling
Edit Greg Hurst answers show how to independently get 3x speed-up using single-threaded MKL implementation and 4x speed-up by using native implementation with multi-threading, can they be combined?
...
18
votes
3
answers
2k
views
How to solve the Monty Hall problem?
Problem
For purely recreational purposes I would like to solve the Monty Hall problem with Mathematica using the function Probability (dedicated to the calculation ...
17
votes
4
answers
5k
views
Random letters: What is the chance that real words are generated?
Suppose that we are just typing random letters with our keyboards and we include space bar. We don't distinguish between capital and lowercase letters. If we do this for a long time, how many of the ...
17
votes
2
answers
13k
views
Performing a chi-square goodness of fit test
How would I perform a chi-square goodness of fit test? I have tried the following, where my data consists of the observed values whilst the data pair contains the correct values. Then I use the ...
17
votes
4
answers
1k
views
Does Mathematica have an implementation of the Poisson binomial distribution?
I need to work out the probability of having $k$ successful trials out of a total of $n$ when success probabilities are heterogeneous. This calculation relates to the Poisson Binomial Distribution. ...
17
votes
1
answer
721
views
Estimating the probability distribution of a Brownian motion particle
I have system of hypothetical physics. A particle traverses a Brownian motion path in free space, and if it hits a boundary, in my case a box with an opening on one side, it teleports to an uniformly ...
17
votes
3
answers
5k
views
Finding distribution parameters of a gaussian mixture distribution
Short version: how to estimate the parameters of a mixture of multivariate normal distributions (i.e.: Gaussian mixture model)?
Long version.
I am trying to estimate the parameters of a mixture of ...
17
votes
2
answers
1k
views
Diffusion probabilistic model in deep generative modeling
Diffusion Models (http://proceedings.mlr.press/v37/sohl-dickstein15.html and https://arxiv.org/abs/2006.11239) are generative models gaining popularity in the community of deep generative modeling, ...
16
votes
2
answers
9k
views
How to calculate the mode of a probability distribution
I was wondering if there is any command to get the mode of the probability distributions. Calculating manually, the mode is (alpha-1)/(alpha+beta-2)
While the mean, median or variance of the beta ...
16
votes
3
answers
914
views
Measure a DensityHistogram[] pair similarity
I study human vision and more specifically eye-movements.
"If we display 2 symmetrical patterns (20 min one after the other), will our gaze distribution be symmetric is my research question."
The 2 ...
16
votes
3
answers
1k
views
Simulate a simple spinner
I'm teaching some simple ideas in probability to students in grade 7.
The "spinner" below works, but I'm wondering how I could make it just a bit more realistic by having it actually "spin" around a ...