1
$\begingroup$

I need to shift one distribution to another using a simple positional correction parameter. For illustrative purposes, I will create a "double hump" or "twin peaks" distribution here, but my actual problem will be a much more random distribution. As such, I do indeed need to use something like SmoothKernelDistribution

Here's the example distribution:

data1 = RandomVariate[NormalDistribution[5, 1.2], 100000];
data2 = RandomVariate[NormalDistribution[10, 1.2], 100000];
dataT = Join[data1, data2];
data3 = RandomVariate[NormalDistribution[6, 1.2], 100000];
data4 = RandomVariate[NormalDistribution[11, 1.2], 100000];
dataT2 = Join[data3, data4];
dataT = DeleteCases[dataT, x_ /; x <= 0];
dataT = DeleteCases[dataT, x_ /; x >= 15];
dataT2 = DeleteCases[dataT2, x_ /; x <= 0];
dataT2 = DeleteCases[dataT2, x_ /; x >= 15];
skd1 = SmoothKernelDistribution[dataT, 
   Automatic, {"Bounded", {0, 15}, "Gaussian"}];
skd2 = SmoothKernelDistribution[dataT2, 
   Automatic, {"Bounded", {0, 15}, "Gaussian"}];
Show[Plot[{PDF[skd1, x], PDF[skd2, x]}, {x, 0, 15}], 
 Histogram[{dataT, dataT2}, Automatic, "PDF"]]

Output: enter image description here

I know I can shift the PDF of dataT2 by -1 to get (roughly) the same PDF. Here it is shifted by 1.1 so show how close it is:

Show[Plot[{PDF[skd1, x], PDF[skd2, x], PDF[skd2, x + 1.1]}, {x, 0, 15}], Histogram[{dataT, dataT2}, Automatic, "PDF"]]

enter image description here

Now I want to tell Mathematica to find the shifting parameter (I used 1.1 above) that best fits the seconds data set to the first. The following code is what I'm using, but it has been running for over an hour so I'm guessing I'm doing something wrong, or there's a much better way. Can anyone please help me get the result I want better/faster?

custom[a_] = 
  ProbabilityDistribution[PDF[skd1, x + a], {x, 0, Infinity}, 
   Assumptions -> a > 0];
Parallelize[
 FindDistributionParameters[dataT, custom[a], 
   ParameterEstimator -> {"MaximumLikelihood", 
     Method -> {"FindMaximum"}}]
  FindDistributionParameters[dataT2, custom[a], 
   ParameterEstimator -> {"MaximumLikelihood", 
     Method -> {"FindMaximum"}}]]
$\endgroup$
  • $\begingroup$ Take the difference in the means and you're done. $\endgroup$ – JimB Sep 26 '17 at 23:52
  • $\begingroup$ If the real distribution I'm working with was just a simple Gaussian like this, you're absolutely right. But it's not... It's a weird distribution that does not have analytical representation. What I posted above was just an example meant to illustrate the problem. $\endgroup$ – Matt Stein Sep 27 '17 at 0:41
  • 1
    $\begingroup$ It doesn't matter what distribution you're working with. If there is a constant shift that distinguishes the difference between the two distributions, then the difference in the sample means, medians, 93rd percentiles, etc. all estimate the same shift. You probably want to choose the estimator with the smallest variance (which is likely the mean or the median). $\endgroup$ – JimB Sep 27 '17 at 1:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.