2
$\begingroup$

Let us consider the following lists

SeedRandom[1]

list1 = RandomReal[{-10, 10}, 1000];
list2 = RandomReal[{-10, 10}, 200];
Histogram[{list1, list2}, 100]

The histogram is the following

enter image description here

I would like to generate a histogram for list1 and normalize each bin by the sum of the counts of list1and list2. So, for example, if in the bin 0 there are 7 points of list1and 2 points of list2, I would like the bin 0 of the new histogram to have the value 7/(7+2).

How can I do this?

$\endgroup$
1
  • $\begingroup$ If you're ending up with very many 0 divided by 0 bins, then what you're proposing sounds like a bad idea. Maybe you could elaborate on why you want to normalize each bin by the sum of the counts. $\endgroup$
    – JimB
    Feb 19, 2022 at 21:51

3 Answers 3

3
$\begingroup$

You can first precompute a histogram heights for both lists with HistogramList, then use this data as an input to a custom hspec function of Histogram (3rd argument).

If there are no elements in a bin, an error message (Power::infy) is thrown. Depending on your personal code aesthetics, you can either appropriately filter out these values or simply turn off the messagey with Quiet, which will also automatically make those bins disappear.

SeedRandom[1];
list1 = RandomReal[{-10, 10}, 100];
list2 = RandomReal[{-10, 10}, 20];

bspec = {-10, 10, 20/60};

totalcounts = Total@(Last@HistogramList[#, bspec] & /@ {list1, list2});

Histogram[{list1, list2}, bspec, Quiet[#2/totalcounts] &, ChartLayout -> "Stacked"]

Histogram

$\endgroup$
2
  • $\begingroup$ In my real dataset there are bins where there is no statistics. How can i avoid the 1/0 error? $\endgroup$
    – apt45
    Feb 19, 2022 at 21:17
  • $\begingroup$ @apt45, please see my update. $\endgroup$
    – Domen
    Feb 19, 2022 at 21:35
2
$\begingroup$

For each bin the ratio of the counts for list1 to the total counts of list1 and list2 has a great deal of sampling variability with the extreme case of having some bins with the ratio of 0 over 0.

If the true probability densities (pdf1 and pdf1 were known, then maybe what corresponds to that ratio is n1*pdf1/(n1*pdf1 + n2*pdf2). In other words you need to state if you had the true pdfs, what would be the summary statistics of interest. What is of interest should be definable in terms of the parameters of the distributions rather than something constructed solely from the sample data.

(* Take random samples from 2 distributions *)
SeedRandom[1234]
n1 = 1000;
n2 = 200;
list1 = RandomReal[{-10, 10}, n1];
list2 = RandomReal[{-10, 10}, n2];

So the first step is to construct an estimate of the two pdf's. The "Bounded" option is used because I'm assuming you really know that the only positive probability density is between -10 and 10.

skd1 = SmoothKernelDistribution[list1, Automatic, {"Bounded", {-10, 10}, "Gaussian"}];
skd2 = SmoothKernelDistribution[list2, Automatic, {"Bounded", {-10, 10}, "Gaussian"}];

Now construct an estimate of the desired function of the true parameters.

Plot[{n1 PDF[skd1, z]/(n1 PDF[skd1, z] + n2 PDF[skd2, z])}, {z, -10, 10}, 
  PlotRange -> {{-10, 10}, {0, 1}}]

Function of parameters of interest

In short, in the 21st century (and even in the late 20th century) you should probably drop the construction of histograms and move on to nonparametric density estimates (and associated functions).

$\endgroup$
1
$\begingroup$

An alternative approach: Post-process Histogram output to modify rectangle coordinates as desired:

ClearAll[yMAX, postProcess]
yMAX = GroupBy[Cases[#, _Rectangle, All], #[[1, 1]]&, Max[{#[[-1]], #2[[-1]]}& @@@ #]&]&

postProcess = ReplaceAll[#, 
        Rectangle[{x1_, y1_}, {x2_, y2_}, c___] :> 
          Rectangle[{x1, y1 / yMAX[#][x1]}, {x2, y2 / yMAX[#][x1]}, c]] &;

Using Domen's example data:

SeedRandom[1];
list1 = RandomReal[{-10, 10}, 100];
list2 = RandomReal[{-10, 10}, 20];

bspec = {-10, 10, 20/60};

hist = Histogram[{list1, list2}, bspec, ChartLayout -> "Stacked", ImageSize -> Large]

enter image description here

postProcess @  hist

enter image description here

Alternatively, we can do the post-processing inside Histogram[...] using the option DisplayFunction -> (postProcess @ # &):

Histogram[{list1, list2}, bspec, ChartLayout -> "Stacked", 
  ImageSize -> Large, DisplayFunction -> (postProcess @ # &)]

same picture

An example with more than two data sets:

postProcess @ Histogram[{list1, list2, list1, list1, list2, list1},
   bspec, ChartLayout -> "Stacked", ImageSize -> Large]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.