2
$\begingroup$

I have a large dataset and I'm trying to plot its scatterplot as well as an overlayed set of binned "summary" data points (in a different color of course). My intended effect is similar to this one

So far, I know how to sort my data's x-y matrix by the x-variable. How can I make it so that I cut this huge matrix into bins of equal width in the x-direction, and then find the average for all the y values within each bin? I feel like this task will be trivial once I can automatically slice my dataset into bins of arbitrary width in the x-direction.

$\endgroup$
  • $\begingroup$ Is this question related to Mathematica ? $\endgroup$ – Sektor Jun 18 '15 at 15:17
  • 2
    $\begingroup$ Histogram and HistogramList $\endgroup$ – BlacKow Jun 18 '15 at 15:26
  • $\begingroup$ How will Histogram work if my goal is to bin it as a scatterplot? $\endgroup$ – joetheschmoe4000 Jun 18 '15 at 15:47
  • $\begingroup$ This is Mathematica related because I want to understand matrix manipulation. So far, I have a list of ordered pairs that looks something like: {{1,2} , {3,4}, {5,6} ....... } (with different values of course). I want to cut this list into bins of any given length, and then average the y-values within each bin. $\endgroup$ – joetheschmoe4000 Jun 18 '15 at 15:52
  • $\begingroup$ You might want to look at BinLists. $\endgroup$ – image_doctor Jun 18 '15 at 15:54
1
$\begingroup$

Let's generate some random data to play with:

data = Table[{x, 0.18 x + 1 + RandomReal[RandomReal[40]]}, {x, 0, 25, 0.01}];
ListPlot[data, PlotRange -> All]

Mathematica graphics

Let us then define a helper function that will calculate the bin-scatter data, given the data and how many bins we want:

Clear[binscatter]
binscatter[list_?ListQ, numbins_Integer] := Module[
  {sorted, partitioned},
  sorted = SortBy[First][list];
  partitioned = Partition[sorted, Floor[Length[sorted]/numbins]];
  Mean /@ partitioned
  ]

We can try this out on the data generated above, by generating e.g. 10 bins:

binscatter[data, 10]

(* Out:
{{1.245, 10.869}, {3.745, 11.9407}, {6.245, 11.3557}, {8.745, 12.4129}, {11.245, 12.8107},
  {13.745, 14.8622}, {16.245, 14.0835}, {18.745, 15.305}, {21.245, 13.7489}, 
  {23.745, 15.7441}}    
*)

We can then plot the bin-scatter data, e.g. on top of the original data, for comparison:

ListPlot[
  {data, binscatter[data, 10]}, 
  PlotStyle -> {Black, Directive[Red, PointSize[0.02]],}
]

bin-scatter plot with original data as well

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.