New answers tagged

0

The discreteness imposed by the arbitrary parameters of a Histogram seems an unnecessary burden to me. I would do ed = EmpiricalDistribution[data]; Plot[CDF[ed, x], {x, 0, 400}, Exclusions -> None] with no arbitrary smoothness and discontinuities in the slope other than the one imposed by the data itself.


5

This answer is based on Brett Champion's comment. The gaps can be removed by giving the option Exclusions -> None. SeedRandom[1]; data = RandomReal[400, 200]; Plot[PDF[HistogramDistribution[data, 20], x], {x, 0, 400}] Plot[CDF[HistogramDistribution[data, 20], x], {x, 0, 400}, Exclusions -> None]


3

The troublesome part here is Abs is not symbolically differentiable: D[Abs[c], c] % /. c -> 1 Abs'[c] Abs'[1] (* doesn't make sense *) To work around this, one way is to use ND instead of D: Needs["NumericalCalculus`"] ND[Abs[a + b I], a, 1] /. b -> 1 0.707107 Another way is to use ComplexExpand before differentiation: D[Abs[a + b I] // ...


5

I am afraid that this is not really an answer, but a collection of bookmarks for future reference, since this question is bound to come up in searches about LOESS and LOWESS on this site. Here are a few implementations found searching the web: @Rahul has volunteered an implementation in an answer on this site: ...


8

It seems that Non-Negative Matrix Factorization (NNMF) can be applied for doing ICA. At least in some cases. In order to demonstrate this I will make up some data in the spirit of the "cocktail party problem". Then I am going to apply an NNMF algorithm. To be clear, NNMF does dimension reduction, but its norm minimization process does not enforce variable ...


5

OK, a partial answer without considering h["TestDataTable",All]: In the plot it is much faster to move the pars inside CDF. (Computing CDF numerically vs. symbolically, I guess.) AbsoluteTiming[d1 = CDF[f[a, b, c, p] /. pars, x];] (* <1s *) AbsoluteTiming[d2 = CDF[f[a, b, c, p], x] /. pars;] (* 14s *) And it is faster to first compute the functions ...


6

For example: xa = Array[x[Sort@{##}] &, {3, 3}]; k = TransformedDistribution[Det@xa, Thread[Union@Flatten[xa] \[Distributed] BernoulliDistribution[1/2]]]; MatrixForm@xa And then Probability[x > 1/4, x \[Distributed] k] Variance[k] Expectation[x, x \[Distributed] k] (* 5/64 35/64 -(3/8) *)


0

The entropy of a normalized list of probabilities is returned by entropy[prob_List]/;Total[prob]==1 := With[{q=prob/.{0->1,0.0->1}}, -q.Log[q] ] This expression avoids 0*Log[0] = Indeterminate results from probability distributions as e.g. {0.0, 0.2, 0.8}.


0

Let $U,V\sim U\left(0,\,1\right)$ be two iid standard uniform random variables. Sample $n$ times from $U$ and denote the sum by $s_{x} := \sum_{k=1}^{n} u_{k}$. Note that this sum $s_{1}$ has the Irwin–Hall distribution, which is defined to be the sum of iid standard uniform random variables. I understand your question in the following way: You are asking ...


0

Here is an approach to produce a good approximation. Brute force generate lots of distributions until we achieve the desired total: target = Total@RandomReal[1, {1000}] 511.315 set2 = NestWhile[ Append[ Rest@#, RandomReal[1]] &, RandomReal[1, {1000}], Abs[Total[#] - target] > .0001 &]; Total@set2 511.315 % - target ...


0

Someone can maybe implement the following idea: It is enough to be able to produce a list of $n$ random numbers in range $[0,1]$ with total sum $s$, where the random numbers are chosen in some uniform (fair) fashion. Note that picking a random vector in $C=[0,1]^n$ is the same as sampling a hypercube. We can intersect this hypercube with the plane ...


6

If you want two lists to have the same Total, then you need to scale one of them by the right amount. The trick is to pick which one to scale so that both of the lists are within $U(0,1)$ n=2000; lists = RandomReal[1, {n, 2}] // Transpose; lists = lists (Min[Total /@ lists]/Total@# & /@ lists); Now you verify that they are both from the right ...


3

General The data has heteroscedastic variables. For heteroscedastic data Quantile regression can be very useful. The blog post "Estimation of conditional density distributions" has analysis description that might be the answer of: The $e_i$ are independent noise from a distribution that depends on $x$ as well as on global parameters of the data; ...


2

This is what I imagined: L = ToExpression@Import["http://pastebin.com/raw/QQ926Qkw"]; design = Transpose[SparseArray[With[{sz = Length /@ L}, With[{sz2 = Accumulate[sz]}, Join[PadRight[MapThread[ Join[ConstantArray[0, #], #2] &, {Prepend[Most[sz2], 0], ConstantArray[1, #] & /@ sz}]], ...


7

It seems Mathematica's Entropy is equivalent to the following code (at least for lists of symbols and strings): entropy[list_List] := With[{p = Tally[list][[All, 2]]/Length[list]}, -p.Log[p] ] entropy[str_String] := With[{p = Tally[Characters@str][[All, 2]]/StringLength[str]}, -p.Log[p] ] You can try this on the examples on the Entropy help ...


3

The Entropy function takes a list of numbers and gets the proportion of values for each unique number and applies the entropy formula you show using those proportions ($p_i$). For a binomial distribution: (* Sample size *) n = 97 (* Take random sample *) x = RandomVariate[BinomialDistribution[1, 0.5], n] (* ...


6

Borrowing from Sjoerd C. de Vries,(noticed this also matches rojolalalalalalalalalalalalala's comment), you don't need to generate a list of random number in order to calculate the entropy of a distribution, but you do need to if you want to use Entropy. Expectation[-Log[PDF[BernoulliDistribution[.2], q]], q \[Distributed] BernoulliDistribution[.2]] (* ...


4

1. Yahtzee Problem For throwing a single die we have x, the number shown on the thrown die, follow a discrete uniform distribution: dist = DiscreteUniformDistribution[ {1, 6} ] Now the probability for getting two fives in the next throw is: Probability[ x1 == 5 && x2 == 5, { x1 \[Distributed] dist, x2 \[Distributed] dist } ] $\frac{1}{36}$ ...


2

I have previously used the following two helper functions to generate the format of covariance and correlation matrices: covariancematrix[n_] := Table[ σ[i] σ[j] ρ[i, j]^(1 - KroneckerDelta[i, j]), {i, 1, n, 1}, {j, 1, n, 1} ] /. {ρ[i_, j_] :> ρ[j, i] /; i > j} correlationmatrix[n_] := Table[ σ[i] σ[j] ρ[i, j]^(1 - KroneckerDelta[i, ...


1

n = 0; a = 1; L = 0; m = 1/2; b = 1; g = (2 m a r)/(n + L + 1); Fi[r_] = (r)^(L + 1) Exp[-((m a)/(n + L + 1)) r] * LaguerreL[n, 2 L + 1, g]; f[r_] = AiryAi[(2 m b)^(1/3) (r)]; mu1 = Integrate[r^(L + 2) f[r], {r, 0, Infinity}]/ Integrate[r^(L + 1) f[r], {r, 0, Infinity}] (* Gamma[1/3]/(3^(1/3)*Gamma[2/3]) *) mu1 // N (* 1.37172 *) mu2 = ...


0

Let's look what we are actually given to work with. n = 0; a = 1; L = 0; m = 1/2; b = 1; g = (2 m a r)/(n + L + 1) r Fi[r_] = r^(L + 1) Exp[-((m a)/(n + L + 1)) r] LaguerreL[n, 2 L + 1, g] r/E^(r/2) f[r_] = AiryAi[(2 m b)^(1/3) r] AiryAi[r] So the means should be give by mean1 = NIntegrate[r AiryAi[r], {r, 0, 6}]/6 0.0431324 ...


4

Mathematica V10 introduced the following two functions: - HiddenMarkovProcess - FindHiddenMarkovStates Examples of their usage can be found here and here.


12

The documentation on TransformedDistribution will guide you through. d = TransformedDistribution[Cos[x], x \[Distributed] UniformDistribution[{0, 2 Pi}]]; pdf = PDF[d, x] Output will be this $$\begin{array}{cc} f_y(x)=\{ & \begin{array}{cc} \text{Indeterminate} & x=-1\lor x=1 \\ \frac{1}{\pi \sqrt{1-x^2}} & -1<x<1 \\ \end{array} ...


4

My interpretation of the question is that, after fixing the error in the definition of the divergence, the mathematical problem that is being asked about arises when the numerator under the Log goes to zero - not (as is written in the question) the denominator. Here is how you can then get the correct result in this case: pmfA = {1/6, 1/6, 1/6, 1/6, 1/6, ...


2

Explicitly telling Mathematica that b is real and positive seems to help. Assuming[b > 0, X = UniformDistribution[{-b, b}]; Z = TransformedDistribution[X^2, X \[Distributed] UniformDistribution[{-b, b}]]; PDF[Z, x] ] Assuming[b > 0, XZ = ProductDistribution[X, Z]; pdf = PDF[XZ, {x, z}] ] Block[{b = 1}, Plot3D[pdf, {x, -1, 1}, {z, -1, ...


2

Since the joint distribution function is singular (proportional to $\delta(y - x^2)$ or something like that), you'll have to be creative about plotting it. Here's an attempt that's kludgey as hell, but it does give something: b = 1; ListPointPlot3D[Table[{x, x^2, PDF[Z, x] PDF[X, x]}, {x, -2 b, 2 b, b/100}], RegionFunction -> Function[{x, y, z}, z ...


1

If you start with $b = 1$ (for instance), you get (Mathematica version 10.3): $\begin{cases} \text{Indeterminate} & x=0\lor x=1 \\ \frac{1}{2 \sqrt{x}} & 0<x<1 \end{cases}$ Which seems perfect. Plot3D[ 1/(4 Sqrt[x]), {x, -1, 1}, {y, -1, 1}]


2

To fit a model and obtain a measure of precision for the parameter estimates one needs to consider the error structure in addition to matching the data with a function to be minimized or maximized. And finding the value of a parameter that minimizes some function does not necessarily result in a “best fit.” Again, it depends on what is a reasonable error ...


2

not an answer. i just wanted to paste output to show the bug has been fixed on Mac


7

Stealing Pickett's example: Values @ GroupBy[res, Query[{"p1", "p2"}], Merge@Mean] { <|"p1" -> 1, "p2" -> 1, "r1" -> 64/5, "r2" -> 20|>, <|"p1" -> 1, "p2" -> 2, "r1" -> 64/5, "r2" -> 156/5|> } And if m keys are required: <|KeyDrop[#, {"r1", "r2"}], "m1" -> #r1, "m2" -> #r2|> & /@ %


2

Update borrowing from Kuba. Join @@@ Normal@GroupBy[dat, Query[{"p1", "p2"}], KeyMap[StringReplace[#, "r" -> "m"] &]@*Mean@*Query[All, {"r1", "r2"}]] Original post Generate some data: SeedRandom[123]; dat = Join[#, <|"r1" -> RandomReal[10], "r2" -> RandomReal[20]|>] & /@ Flatten@ConstantArray[Table[<|"p1" -> i, "p2" ...


3

One way: gathered = GatherBy[res, {#p1, #p2} &] <| "p1" -> First[#]["p1"], "p2" -> First[#]["p2"], Merge[#[[All, {Key["r1"], Key["r2"]}]], Mean] |> & /@ gathered Test: res = { <|"p1" -> 1, "p2" -> 1, "r1" -> 10, "r2" -> 20|>, <|"p1" -> 1, "p2" -> 1, "r1" -> 15, "r2" -> 20|>, ...



Top 50 recent answers are included