Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Quantile[] is the workhorse method in robust data analysis and statistics (see, eg Koenker's Quantile Regression). However, it should be complemented by an InverseQuantile[] method that ideally satisfies the equations InverseQuantile[data,Quantile[data,q]]==q (0 <= q <= 1) and Quantile[data,InverseQuantile[data,v]]==v, where either MemberQ[data,v]==True or perhaps Min[data] <= v <= Max[data]. The reason for the inverse is that in addition to computing median, interquartile ranges and so on, it's often necessary to test whether a data value is above or below median etc (eg, to color-scale in visualization)

There are 2 immediate problems, invertibility and optional parameters and their interaction:

  1. Quantile is not injective when considered as a function of q real number, eg Quantile[{1, 2, 2, 4}, 1/3] == Quantile[{1, 2, 2, 4}, 2/3] == 2. The obvious solution is to define Quantile[] and InverseQuantile[] to return an Interval value instead of a real; this interval is the solution of a constrained optimization problem since it should be the maximal interval consistent with the parameters.

  2. Quantile is also partially defined by an optional 4-dimensional parameter space, whose default value is {{0,0},{0,1}} - corresponding to "inverse empirical CDF" method. 7 other specific parameter values listed in MMA8 docs are named methods, while other values presumably correspond to hybrid or interpolated methods. Comparing the behavior of Quantile[] on model data {1,2,2,4} versus Table[RandomReal[], {100}], the choice of optional parameters seems to have a greater effect in the former, which represents data with tie values, eg:

    ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
      Filling -> Bottom] &@Table[Table[
       Quantile[Table[RandomReal[], {100}], 
       q, {{a = 1, b = 0}, {c, d = 0}}],
      {q, 0, 1, 1/10}],
     {c, 0, 1, 1/20}]
    

    versus

    ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
      Filling -> Bottom] &@Table[Table[
       Quantile[{1, 2, 2, 4}, q, {{a = 0, b}, {c = 0, d = 1}}],
      {q, 0, 1, 1/10}],
     {b, -3, 3, 1/20}]
    

How can an InverseQuantile function be implemented to return an interval instead of a single number?

share|improve this question
add comment

1 Answer 1

up vote 3 down vote accepted

Something like this should be close:

InverseQuantile[list_, x_] := LengthWhile[list // Sort, # <= x &]/Length[list]

Depending on your definition of quantiles you may choose to have the test be # < x & or # <= x &

This is with # <= x &:

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

(* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 1}, 
 {1.7, 1}, {1.8, 1}, {1.9, 1}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2}, 
 {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2}, 
 {3.1, 2}, {3.2, 2}, {3.3, 2}, {3.4, 2}, {3.5, 2}, {3.6, 2}, {3.7, 2}, 
 {3.8, 2}, {3.9, 2}, {4., 4}} *)

or, with Interval:

InverseQuantile[list_, x_] := 
 Interval[
  {
   LengthWhile[list // Sort, # < x &]/Length[list], 
   LengthWhile[list // Sort, # <= x &]/Length[list]
  }
 ]

InverseQuantile[{1, 2, 2, 4}, #] & /@ {0, 0.5, 1, 1.5, 2, 3, 4, 5}
(* {Interval[{0, 0}], Interval[{0, 0}], Interval[{0, 1/4}], Interval[{1/4, 1/4}], 
    Interval[{1/4, 3/4}], Interval[{3/4, 3/4}], Interval[{3/4, 1}], Interval[{1, 1}]} *)

Another approach would be to base InverseQuantile on empirical distributions and their CDF. Two possibilities are:

InverseQuantile[list_, x_] := CDF[SmoothKernelDistribution[list], x]

InverseQuantile[list_, x_] := CDF[HistogramDistribution[list], x]

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

    (* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 2},
            {1.7, 2}, {1.8, 2}, {1.9, 2}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2},
            {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2},
            {3.1, 2}, {3.2, 2}, {3.3, 4}, {3.4, 4}, {3.5, 4}, {3.6, 4}, {3.7, 4}, 
            {3.8, 4}, {3.9, 4}, {4., 4}} *)
share|improve this answer
    
I've tested your 1st implementation using InverseQuantile above, and upvoted for your effort, but can't replicate the expected identity relation: Table[{x,Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, {x, 1, 4,0.1}] yields the numbers 1 and 2 as opposed to the original input argument x. –  alancalvitti May 30 '12 at 20:47
1  
@alancalvitti As I wrote earlier, it all depends on some choices you make. If you change the test # < x & to # <= x & it works as you wish. I added two new methods too. –  Sjoerd C. de Vries May 30 '12 at 21:37
    
Thank you, great use of CDF. Yet to be addressed are (1) interaction with the 4-dimensional parameter space of Quantile and (2) IntervalValued Quantile. –  alancalvitti May 31 '12 at 13:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.