3
$\begingroup$

Given a histogram of some data, such as e.g. in the following sample

https://www.codepile.net/pile/NLlNgP60

sample = Import["https://pastebin.com/raw/yFJa87Hn", "RawJSON"];

I can make a histogram:

Histogram[sample, 100]

enter image description here

I am looking for ways to automatically find bounds on the histogram distributions, such as I manually added in the following picture:

enter image description here

Note that the bounds are a bit fuzzy, strictly outside the distribution when the edge is sharp and high, but reaching a bit inside the distribution when the falloff has a soft "tail".

Can this be done in Mathematica? What would be a good approach?

$\endgroup$
  • 2
    $\begingroup$ Can you elaborate on the data collection process and what a penalty occurs when an area of low density is missed? I ask because otherwise this becomes a subject-matter-free data manipulation where the "answer" can be "I'll know it when I see it." The two current answers below certainly do well at the heavy-lifting part of sorting out potential groups. I'm curious as to what you do with the potential suspects once they're in the line up. $\endgroup$ – JimB Aug 8 at 0:09
7
$\begingroup$

One option could be to use FindClusters and MinMax or Quantile

sample = Import["https://pastebin.com/raw/yFJa87Hn", "RawJSON"];

clusterEdges = MinMax /@ FindClusters[sample, 3]
(* {{1.00194, 1.0076}, {0.988321, 0.994815}, {0.995332, 1.00138}} *)

clusterEdges2 = Quantile[#, {0.1, 0.9}] & /@ FindClusters[sample, 3]
(* {{1.00319, 1.00651}, {0.989299, 0.994465}, {0.997048, 
  0.999483}} *)


Histogram[
 sample
 , 100
 , Epilog -> {
   Thick,
    Black, Map[ Line[Transpose[{#, {10, 10}}]] &, clusterEdges],
    Red, Map[ Line[Transpose[{#, {15, 15}}]] &, clusterEdges2]
   }
 , PlotTheme -> "Scientific"
 ]

enter image description here

$\endgroup$
4
$\begingroup$

Since you're working with pretty dense histogram data you can also play with the smooth-kernel density estimator form of your histogram:

SmoothHistogram[sample]

enter image description here

dist = SmoothKernelDistribution[sample];
cddd = PDF[dist, x];
boundary1 = FindMinimum[{cddd, x > .995, x < 1}, {x, .995}] // Quiet
boundary2 = FindMinimum[{cddd, x > 1, x < 1.005}, {x, 1}] // Quiet

{23.8826, {x -> 0.995819}}

{18.3624, {x -> 1.00092}}

You can also play with the density estimator to find a more precise cut-point:

SmoothHistogram[sample, .0002]

enter image description here

dist2 = SmoothKernelDistribution[sample, .0002];
cddd2 = PDF[dist2, x];
boundary12 = FindMinimum[{cddd2, x > .995, x < 1}, {x, .995}] // Quiet
boundary12 = FindMinimum[{cddd2, x > 1, x < 1.005}, {x, 1}] // Quiet

{2.54235, {x -> 0.995272}}

{0.352377, {x -> 1.00102}}

If you know a good way to get FindMinimum to return all the minima within a given range that'll work better than plot-and-guess starting conditions

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.