2
$\begingroup$

Quantile[] is the workhorse method in robust data analysis and statistics (see, eg Koenker's Quantile Regression). However, it should be complemented by an InverseQuantile[] method that ideally satisfies the equations InverseQuantile[data,Quantile[data,q]]==q (0 <= q <= 1) and Quantile[data,InverseQuantile[data,v]]==v, where either MemberQ[data,v]==True or perhaps Min[data] <= v <= Max[data]. The reason for the inverse is that in addition to computing median, interquartile ranges and so on, it's often necessary to test whether a data value is above or below median etc (eg, to color-scale in visualization)

There are 2 immediate problems, invertibility and optional parameters and their interaction:

  1. Quantile is not injective when considered as a function of q real number, eg Quantile[{1, 2, 2, 4}, 1/3] == Quantile[{1, 2, 2, 4}, 2/3] == 2. The obvious solution is to define Quantile[] and InverseQuantile[] to return an Interval value instead of a real; this interval is the solution of a constrained optimization problem since it should be the maximal interval consistent with the parameters.

  2. Quantile is also partially defined by an optional 4-dimensional parameter space, whose default value is {{0,0},{0,1}} - corresponding to "inverse empirical CDF" method. 7 other specific parameter values listed in MMA8 docs are named methods, while other values presumably correspond to hybrid or interpolated methods. Comparing the behavior of Quantile[] on model data {1,2,2,4} versus Table[RandomReal[], {100}], the choice of optional parameters seems to have a greater effect in the former, which represents data with tie values, eg:

    ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
      Filling -> Bottom] &@Table[Table[
       Quantile[Table[RandomReal[], {100}], 
       q, {{a = 1, b = 0}, {c, d = 0}}],
      {q, 0, 1, 1/10}],
     {c, 0, 1, 1/20}]
    

    versus

    ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
      Filling -> Bottom] &@Table[Table[
       Quantile[{1, 2, 2, 4}, q, {{a = 0, b}, {c = 0, d = 1}}],
      {q, 0, 1, 1/10}],
     {b, -3, 3, 1/20}]
    

How can an InverseQuantile function be implemented to return an interval instead of a single number?

$\endgroup$
0

1 Answer 1

4
$\begingroup$

Something like this should be close:

InverseQuantile[list_, x_] := LengthWhile[list // Sort, # <= x &]/Length[list]

Depending on your definition of quantiles you may choose to have the test be # < x & or # <= x &

This is with # <= x &:

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

(* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 1}, 
 {1.7, 1}, {1.8, 1}, {1.9, 1}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2}, 
 {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2}, 
 {3.1, 2}, {3.2, 2}, {3.3, 2}, {3.4, 2}, {3.5, 2}, {3.6, 2}, {3.7, 2}, 
 {3.8, 2}, {3.9, 2}, {4., 4}} *)

or, with Interval:

InverseQuantile[list_, x_] := 
 Interval[
  {
   LengthWhile[list // Sort, # < x &]/Length[list], 
   LengthWhile[list // Sort, # <= x &]/Length[list]
  }
 ]

InverseQuantile[{1, 2, 2, 4}, #] & /@ {0, 0.5, 1, 1.5, 2, 3, 4, 5}
(* {Interval[{0, 0}], Interval[{0, 0}], Interval[{0, 1/4}], Interval[{1/4, 1/4}], 
    Interval[{1/4, 3/4}], Interval[{3/4, 3/4}], Interval[{3/4, 1}], Interval[{1, 1}]} *)

Another approach would be to base InverseQuantile on empirical distributions and their CDF. Two possibilities are:

InverseQuantile[list_, x_] := CDF[SmoothKernelDistribution[list], x]

InverseQuantile[list_, x_] := CDF[HistogramDistribution[list], x]

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

    (* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 2},
            {1.7, 2}, {1.8, 2}, {1.9, 2}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2},
            {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2},
            {3.1, 2}, {3.2, 2}, {3.3, 4}, {3.4, 4}, {3.5, 4}, {3.6, 4}, {3.7, 4}, 
            {3.8, 4}, {3.9, 4}, {4., 4}} *)
$\endgroup$
3
  • $\begingroup$ I've tested your 1st implementation using InverseQuantile above, and upvoted for your effort, but can't replicate the expected identity relation: Table[{x,Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, {x, 1, 4,0.1}] yields the numbers 1 and 2 as opposed to the original input argument x. $\endgroup$ May 30, 2012 at 20:47
  • 1
    $\begingroup$ @alancalvitti As I wrote earlier, it all depends on some choices you make. If you change the test # < x & to # <= x & it works as you wish. I added two new methods too. $\endgroup$ May 30, 2012 at 21:37
  • $\begingroup$ Thank you, great use of CDF. Yet to be addressed are (1) interaction with the 4-dimensional parameter space of Quantile and (2) IntervalValued Quantile. $\endgroup$ May 31, 2012 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.