11
$\begingroup$

Goal

The goal is to merge the default styles of a few different charts; namely, to have a chart that:

  • has the key components of BoxWhiskerChart
  • the outline of the DistributionChart's ChartElementFunction$\rightarrow$SmoothDensity
  • has the points of the DistributionChart ChartElementFunction$\rightarrow$PointDensity

Further, I would like to have the ability to:

  • color the points based on other data
  • add Callouts to specified points.

Final outcome addressing all the requirements above should look like:

enter image description here

Example

Sample data

data = {RandomInteger[100, 30], RandomInteger[100, 30]};

Individual chart elements

BoxWhiskerChart[data]
DistributionChart[data]
DistributionChart[data, ChartElementFunction -> "PointDensity"]

External data for color:

With[{perm=Permute[data[[1]]]}, class1 = perm[[1;;Floor@Length@perm/2]]; class2 = perm[[Floor@Length@perm/2+1;;]]]

Such that points belonging to class1 are one color and class2 are another.

enter image description here

Here is an image showing the transformations I would like to make:

enter image description here



Understanding @kglr's code

Kudos to @kglr for being able to do this, but I have very little idea of what is going on in the code. Let's break it down line, by line, and perhaps tweak this a bit for re-use :)


For two dimensional charts, ChartElementFunction passes region, values and metadata to the function. So let's start by changing the function definition to be more readable:

Options[BoxWhiskerDistributionChart] = {"InternalOpacity" -> 0.7};

BoxWhiskerDistributionChart[{{xmin_, xmax_}, {ymin_, ymax_}}, values_,metadata_, OptionsPattern[]] := 
Module[
{parameters = {{{xmin, xmax}, {ymin, ymax}}, values, metadata}},
{    
(*block of code*)
}
]

Yes the definition is longer, but it is more readable to people (like me) who do not dabble with customizing graphic often. The local variable parameters will take the place of ##.


Inside the block of code, the first two lines are:

EdgeForm[{Thick, Charting`ChartStyleInformation["Color"]}],
FaceForm[Opacity[.7, Charting`ChartStyleInformation["Color"]]],

I do not understand why we have to call the Charting` context... I believe, we could tweak this to be:

EdgeForm[Thick],
FaceForm[Opacity[OptionValue["InternalOpacity"]]],

Next we have the first With block:

With[
 {dd = #2},
 Normal[ChartElementDataFunction["SmoothDensity"][##]] /. 
  GeometricTransformation[Polygon[x_, y___], tr_] :> 
   Polygon[DeleteCases[
     AffineTransform[tr]@
      x, {_, _?((# < Quantile[dd, .25] || # > Quantile[dd, .75]) &)}],
     y]
 ]

First we will apply some changes in variables - according to our new function definition - remove some unneeded code, and rename other variables.

ChartElementDataFunction["SmoothDensity"][parameters] /. 
  GeometricTransformation[Polygon[x_, y___], transformation_] :> 
   Polygon[DeleteCases[
     AffineTransform[transformation_]@
      x, {_, _?((# < Quantile[values, .25] || # > Quantile[values, .75]) &)}],
     y]

The first line of this chuck of code:

ChartElementDataFunction["SmoothDensity"][parameters]

Is just the operator form of applying the "SmoothDensity" ChartElementFunction that we want to our current parameter. The Normal was not needed.

Because our new function definition gives us access to the values, we can remove the With block.

My remaining questions on this chuck of the code are as follows:

  1. You have an internally defined function with a delayed rule (see below). What is the transformation function. Where did it come from?

    GeometricTransformation[Polygon[x_, y___], tr_] :> etc

2.) I can see what this does (selects only the data within the .25=.75 quantiles), but I have no idea how:

AffineTransform[transformation_]@x, {_, _?((# < Quantile[values, .25] || # > Quantile[values, .75]) &)}],y]

So we apply the mysterious transformation to only the x values for some reason, then pattern match for any pair such that the y value falls in the right quantile.

This entire line has to have a cleaner way of being written (e.g. write out the variables for y_ _ _, etc. I just don't know what that is...


The next line is straight forward. Nothing to change.


Now we add the points with this chuck:

ChartElementDataFunction["PointDensity", 
   "PointStyle" -> PointSize[Medium]][##] /. 
 GraphicsGroup[{x_, y__}] :> ({y} /. 
    Point[p_] :> (({If[MemberQ[class, #[[2]]], col], Point[{#}]} & /@ 
        p)))

Again we can rewrite parts of it:

ChartElementDataFunction["PointDensity", 
   "PointStyle" -> PointSize[Medium]][parameters] /. 
 GraphicsGroup[{x_, y__}] :> ({y} /. 
    Point[p_] :> (({If[MemberQ[class, #[[2]]], col], Point[{#}]} & /@ 
        p)))

This code isolates which points are of interest and changes their color. However I do not find this code straight forward with two delayed rules and replacement.

So if someone can explain this to me? All of it.


The next two lines, are called to overwrite the "PointDensity" Edge and Face Forms.

While I see that it works. Why do we have to recall them? Can't we just assign them prior to PointDensity and only take the Points?


and the last line is best summed up by @kglr:

Finally, SystemBarFunctionDumpboxplot[] produces the basic box-whisker graphics objects in BoxWhiskerChart, and the modification I made is to add the directives EdgeForm[] and FaceForm[]

Although I do not know why we have to call that context... Is there a way to do this without getting into contexts?



Ideal

The following code outlines a more ideal solution, allowing for users to alter various parameters, change the cutoff, and specify points to highlight via the corresponding option.

This does not work. I think it either has something to do with Module, or the explicit use of the parameters...

Options[BoxWhiskerDistributionChart] = {"InternalOpacity" -> 0.7, 
   "PointsToHighlight" -> {}, "PointToHighlightColor" -> Black, 
   "UpperCutoff" -> .75, "LowerCutoff" -> .25, 
   "EdgeColor" -> Charting`ChartStyleInformation["Color"] };
BoxWhiskerDistributionChart[{{xmin_, xmax_}, {ymin_, ymax_}}, values_,
   metadata_, OptionsPattern[]] :=
 Module[
  {chartParameters = {{{xmin, xmax}, {ymin, ymax}}, values, metadata}},
  {

     (*Give boxes thick edges with their respective color*)

     EdgeForm[{Thick, OptionValue["EdgeColor"]}],
     (*Alter boxes internal opacity with their respective color*)

      FaceForm[
      Opacity[OptionValue["InternalOpacity"], 
       Charting`ChartStyleInformation["Color"]]],

     (*Use smooth density to change box shape, 
     with given quantile cutoffs*)

     ChartElementDataFunction["SmoothDensity"][chartParameters] /. 
      GeometricTransformation[Polygon[x_, y___], transformation_] :> 
       Polygon[
        DeleteCases[
         AffineTransform[transformation]@
          x, {_, _?((# < 
                Quantile[values, OptionValue["LowerCutoff"]] || # > 
                Quantile[values, OptionValue["UpperCutoff"]]) &)}], y],
     Opacity[1],

     (*Color points also foudn in the PointsToHighlight option*)

     ChartElementDataFunction["PointDensity", 
        "PointStyle" -> PointSize[Medium]][chartParameters] /. 
      GraphicsGroup[{x_, y__}] :> ({y} /. 
         Point[p_] :> (({If[
                MemberQ[OptionValue["PointsToHighlight"], #[[2]]], 
                OptionValue["PointToHighlightColor"]], Point[{#}]} & /@
              p))),

     (*Remove the default BoxWhiskerChart EdgeForm*)
     EdgeForm[],
     (*Remove the default BoxWhiskerChart FaceForm*)
     FaceForm[],
     (*Produce the plot*)

     System`BarFunctionDump`boxplot[][chartParameters]
     } &;
  ]
$\endgroup$
  • $\begingroup$ I have sadly currently no time but i brainstormed something for you which should be a helpful start. Graphics[Cases[Level[DistributionChart[data],-1],GeometricTransformationBox[PolygonBox[_],___]],AspectRatio->1] $\endgroup$ – Julien Kluge Oct 8 '16 at 15:45
  • $\begingroup$ SumNeuron, I cut and pasted the picture that summarizes your requirements nicely in the Goal section. If you you don't like it please revert to the previous edit. $\endgroup$ – kglr Aug 23 '17 at 3:27
12
$\begingroup$

Final Update: Adding Callouts to special points

ClearAll[leaderSizeF, calloutF, densityCalloutWhiskerF]
leaderSizeF[bx_] := Switch[Sign[(# - Mean /@ bx)], 
  {1, 1} | {0, 0} |{1, 0} | {0, 1}, {{40, 45 Degree, 0}, {40, 0 Degree}}, 
  {1, -1} | {0, -1}, {{40, 315 Degree, 0}, {40, 0 Degree}},
  {-1, 1} | {-1, 0}, {{40, 135 Degree, 0}, {40, 180 Degree}}, 
  {-1, -1}, {{40, 225 Degree, 0}, {40, 180 Degree}}] &

calloutF[label_ : "Special Point", labelcol_ : Black, fontsize_ : 12, col_ : Red][bx_] := 
 Function[p, Callout[p, Style[ label, fontsize, Bold, labelcol], 
   Appearance -> "Leader", LeaderSize -> (leaderSizeF[bx][p]), 
   CalloutMarker -> "CirclePoint", "CalloutStyle" -> col]]

densityCalloutWhiskerF[label_:"Special Point", labelcol_:Black, 
      fontsize_:12, col_:Red][class1_ : {}, class2_ : {}, from_ : .25, 
      to_ : .75, opacity_ : .7, pointstyle_ : PointSize[Medium], 
      highlightstyle_ : Directive[PointSize[Large], Black]] := 
    Module[{xx = (SeedRandom[123]; RandomReal[#[[1]], Length@class2]),
      color = Charting`ChartStyleInformation["Color"],
      boxrange = {Min[#2], Quantile[#2, from], Median[#2], Quantile[#2, to], Max[#2]}, 
      thresholdrule = GeometricTransformation[Polygon[x_, y___], tr_] :> 
        Polygon[Select[AffineTransform[tr]@x, 
          Function[z, IntervalMemberQ[Interval[Quantile[#2, {from, to}]], z[[2]]]]], y]},
      Charting`ChartStyleInformation["BoxRange"] = boxrange; 
   {EdgeForm[{Thick, color}], FaceForm[Opacity[opacity, color]], 
    ChartElementDataFunction["SmoothDensity"][##] /. thresholdrule,
    EdgeForm[], FaceForm[],  ChartElementDataFunction["BoxWhisker"][##],
    Opacity[1],  ListPlot[ Transpose[{RandomReal[#[[1]], Length@#2], #2}] /. 
       {p : {_, Alternatives @@ class1} :> Style[p, highlightstyle] }, 
       PlotStyle -> Directive[Darker[color], pointstyle]][[1]], 
    ListPlot[calloutF[label, labelcol, fontsize, col][#] /@ 
      Transpose[{xx, class2}]][[1]] /. Rotate[x_, 0.] :> x}] &;

Example:

SeedRandom[1]
data = RandomInteger[100, 100];
class1 = RandomSample[data, Length[data]/4];
class2 = RandomSample[class1, 10];
class1 = Complement[class1, class2];

BoxWhiskerChart[ data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
  ImageSize -> 700, ChartStyle -> 1, PerformanceGoal -> "Speed", 
  ChartElementFunction -> densityCalloutWhiskerF[][class1, class2, .3, .7, .5, 
   PointSize[Medium], Directive[PointSize[.007], Green]]]

enter image description here

Update: Making the hard coded parameters in the original answer optional arguments with default values:

ClearAll[densityWhiskerF]
densityWhiskerF[class_: {}, from_: .25, to_: .75, opacity_: .7,
    pointstyle_: PointSize[Medium], 
    highlightstyle_: Directive[PointSize[Large], Black]] := 
  Module[{color = Charting`ChartStyleInformation["Color"],
     boxrange = {Min[#2], Quantile[#2, from], Median[#2], Quantile[#2, to], Max[#2]},
     thresholdrule = GeometricTransformation[Polygon[x_, y___], tr_] :> 
       Polygon[Select[AffineTransform[tr]@x, 
         Function[z, Quantile[#2, from] <= z[[2]] <= Quantile[#2, to]]], y],
     highlightrule = Point[p_] :> ({If[MemberQ[class, f2@#], highlightstyle], 
       Point[{#}]} & /@ p)}, 
    Charting`ChartStyleInformation["BoxRange"] = boxrange;
    {EdgeForm[{Thick, color}], FaceForm[Opacity[opacity, color]], 
      ChartElementDataFunction["SmoothDensity"][##] /. thresholdrule,
     Opacity[1], ChartElementDataFunction["PointDensity",
          "PointStyle" -> pointstyle][##] /. highlightrule, 
     EdgeForm[], FaceForm[], ChartElementDataFunction["BoxWhisker"][##]}] &;

Examples:

SeedRandom[1]
data = RandomInteger[100, {10, 100}];
perm = RandomSample[data[[1]]];
class1 = perm[[1 ;; Floor@Length@perm/4]];

Legended[BoxWhiskerChart[data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
   ImageSize -> 700, ChartStyle -> 24, ChartElementFunction -> densityWhiskerF[class1]], 
 Placed[PointLegend[{Directive[Black, PointSize[Large]]}, 
   {Style["Special points", 16, "Panel"]}], Above]]

enter image description here

With non-default values:

Legended[BoxWhiskerChart[data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
   ChartStyle -> 24, ImageSize -> 700,
   ChartElementFunction ->  densityWhiskerF[class1, .4, .6, .8, 
     AbsolutePointSize[5], Directive[PointSize[Large], Red]]], 
 Placed[PointLegend[{Directive[Red, PointSize[Large]]}, 
    {Style["Special points", 16, "Panel"]}], Above]]

enter image description here

With default values of the arguments:

Legended[BoxWhiskerChart[data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
  ChartStyle -> 24, ImageSize -> 700, ChartElementFunction -> densityWhiskerF[]], 
 Placed[PointLegend[{Directive[Red, PointSize[Large]]}, 
  {Style["Special points", 16, "Panel"]}], Above]]

enter image description here

Update 2: With a few changes OP's ideal solution can be fixed to give the same result:

ClearAll[BoxWhiskerDistributionChart]
Options[BoxWhiskerDistributionChart] = {"InternalOpacity" -> 0.7, 
   "PointsToHighlight" -> {}, "PointToHighlightColor" -> Black, 
   "UpperCutoff" -> .75, "LowerCutoff" -> .25, 
   "EdgeColor" -> Charting`ChartStyleInformation["Color"]};

BoxWhiskerDistributionChart[{{xmin_, xmax_}, {ymin_, ymax_}}, values_,
   metadata_, OptionsPattern[]] := 
 Module[{chartParameters = Sequence[{{xmin, xmax}, {ymin, ymax}}, values, metadata]}, 
 {(*Give boxes thick edges with their respective color*) 
  EdgeForm[{Thick, OptionValue["EdgeColor"]}],
  (*Alter boxes internal opacity with their respective color*)
   FaceForm[Opacity[OptionValue["InternalOpacity"], 
     Charting`ChartStyleInformation["Color"]]],
  (*Use smooth density to change box shape, with given quantile cutoffs*)
   ChartElementDataFunction["SmoothDensity"][chartParameters] /. 
    GeometricTransformation[Polygon[x_, y___], transformation_] :> 
     Polygon[DeleteCases[AffineTransform[transformation] @ x, 
      {_, _?((# < Quantile[values, OptionValue["LowerCutoff"]] || # > 
              Quantile[values, OptionValue["UpperCutoff"]]) &)}], y],       Opacity[1],
   (*Color points also foudn in the PointsToHighlight option*) 
    ChartElementDataFunction["PointDensity", "PointStyle" ->
     PointSize[Medium]][chartParameters] /. 
      GraphicsGroup[{x_, y__}] :> ({y} /. 
       Point[p_] :> (({If[MemberQ[OptionValue["PointsToHighlight"], #[[2]]], 
              OptionValue["PointToHighlightColor"]], Point[{#}]} & /@ p))),
   (*Remove the default BoxWhiskerChart EdgeForm*)
   EdgeForm[],
   (*Remove the default BoxWhiskerChart FaceForm*)
   FaceForm[],
  (*Produce the plot*)
   System`BarFunctionDump`boxplot[][chartParameters]}]

Example:

BoxWhiskerChart[data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
 ChartStyle -> 24, ImageSize -> 700, ChartElementFunction -> 
  (BoxWhiskerDistributionChart[##, "PointsToHighlight" -> class1] &)]

enter image description here

Original answer:

You can use a custom ChartElementFunction to get almost all your requirements:

ClearAll[ceF]
ceF[class_, col_: Black] := {EdgeForm[{Thick, Charting`ChartStyleInformation["Color"]}], 
     FaceForm[Opacity[.7, Charting`ChartStyleInformation["Color"]]], 
     With[{dd = #2}, Normal[ChartElementDataFunction["SmoothDensity"][##]] /. 
       GeometricTransformation[Polygon[x_, y___], tr_] :> 
        Polygon[DeleteCases[AffineTransform[tr]@x, {_, _?((# < Quantile[dd, .25] || # > 
                 Quantile[dd, .75]) &)}], y]], Opacity[1], 
     ChartElementDataFunction["PointDensity", "PointStyle" -> PointSize[Medium]][##] /. 
      GraphicsGroup[{x_, y__}] :> ({y} /. 
         Point[p_] :> (({If[MemberQ[class, #[[2]]], col], Point[{#}]} & /@ p))), 
     EdgeForm[], FaceForm[], System`BarFunctionDump`boxplot[][##]} &;

data = {RandomInteger[100, 100], RandomInteger[100, 100]};
With[{perm = RandomSample[data[[1]]]}, 
  class1 = perm[[1 ;; Floor@Length@perm/4]];
  class2 = perm[[Floor@Length@perm/4 + 1 ;;]]];

Legended[BoxWhiskerChart[data, {{"MedianMarker", 1, Directive[Thickness[.007], White]}}, 
  ChartStyle -> {Red, Blue}, ChartElementFunction -> ceF[class1]], 
 Placed[PointLegend[{Black}, {"Special points"}], Above]]

"Mathematica graphics"

Notes:

The function ceF modifies and combines several built-in ChartElementFunctions to produce the desired graphics objects, Please see the docs on ChartElementFunction.

ChartElementDataFunction["PointDensity", "PointStyle" -> PointSize[Medium]] is the full name of the function that produces the point density in DistributionChart; I modified it to remove the background density and to change the color of the special points.

Similarly, ChartElementDataFunction["SmoothDensity"] is the function that produces smooth density objects in DistributionChart. I post-processed the output of this function to remove the portions outside the range from .25th to .75th percentiles.

Finally, System`BarFunctionDump`boxplot[] produces the basic box-whisker graphics objects in BoxWhiskerChart, and the modification I made is to add the directives EdgeForm[] and FaceForm[] to effectively remove the box part of the box-whisker graphics object.

$\endgroup$
  • 2
    $\begingroup$ thanks for this useful answer :) $\endgroup$ – ubpdqn Oct 9 '16 at 4:10
  • $\begingroup$ Thank you for the vote @ubpdqn. $\endgroup$ – kglr Oct 9 '16 at 4:15
  • $\begingroup$ @kglr Also thank you for this. You use many aspects that I am very unfamiliar with, e.g. Charting`ChartStyleInformation (what is the tick doing?). I know this will take a substantial bit of time, however I was hoping you might elucidated each step in making your function. $\endgroup$ – SumNeuron Oct 9 '16 at 12:38
  • $\begingroup$ @SumNeuron, for the tick please see tutorial/Contexts. I updated the post with some notes on how the function ceF is constructed. Hope this helps. $\endgroup$ – kglr Oct 9 '16 at 14:42
  • $\begingroup$ @kglr I am reading up on context and CEFs. However you do quite a lot rather quickly. What is the double slot ##? the _?? What is the value that of #2? It isn't the second argument col_. I am sorry for asking so much, however I do really want to understand this. Could you possible make a concrete example showing how each transformation occurs? $\endgroup$ – SumNeuron Oct 9 '16 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.