How can an InverseQuantile or an interval valued Quantile function be implemented in Mathematica?

Question

Quantile[] is the workhorse method in robust data analysis and statistics (see, eg Koenker's Quantile Regression). However, it should be complemented by an InverseQuantile[] method that ideally satisfies the equations InverseQuantile[data,Quantile[data,q]]==q (0 <= q <= 1) and Quantile[data,InverseQuantile[data,v]]==v, where either MemberQ[data,v]==True or perhaps Min[data] <= v <= Max[data]. The reason for the inverse is that in addition to computing median, interquartile ranges and so on, it's often necessary to test whether a data value is above or below median etc (eg, to color-scale in visualization)

There are 2 immediate problems, invertibility and optional parameters and their interaction:

1. Quantile is not injective when considered as a function of q real number, eg Quantile[{1, 2, 2, 4}, 1/3] == Quantile[{1, 2, 2, 4}, 2/3] == 2. The obvious solution is to define Quantile[] and InverseQuantile[] to return an Interval value instead of a real; this interval is the solution of a constrained optimization problem since it should be the maximal interval consistent with the parameters.

2. Quantile is also partially defined by an optional 4-dimensional parameter space, whose default value is {{0,0},{0,1}} - corresponding to "inverse empirical CDF" method. 7 other specific parameter values listed in MMA8 docs are named methods, while other values presumably correspond to hybrid or interpolated methods. Comparing the behavior of Quantile[] on model data {1,2,2,4} versus Table[RandomReal[], {100}], the choice of optional parameters seems to have a greater effect in the former, which represents data with tie values, eg:

   ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
     Filling -> Bottom] &@Table[Table[
      Quantile[Table[RandomReal[], {100}], 
      q, {{a = 1, b = 0}, {c, d = 0}}],
     {q, 0, 1, 1/10}],
    {c, 0, 1, 1/20}]
   

   versus

   ListPlot3D[#, Mesh -> None, InterpolationOrder -> 0, 
     Filling -> Bottom] &@Table[Table[
      Quantile[{1, 2, 2, 4}, q, {{a = 0, b}, {c = 0, d = 1}}],
     {q, 0, 1, 1/10}],
    {b, -3, 3, 1/20}]
   

How can an InverseQuantile function be implemented to return an interval instead of a single number?

Answer 1

Something like this should be close:

InverseQuantile[list_, x_] := LengthWhile[list // Sort, # <= x &]/Length[list]

Depending on your definition of quantiles you may choose to have the test be # < x & or # <= x &

This is with # <= x &:

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

(* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 1}, 
 {1.7, 1}, {1.8, 1}, {1.9, 1}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2}, 
 {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2}, 
 {3.1, 2}, {3.2, 2}, {3.3, 2}, {3.4, 2}, {3.5, 2}, {3.6, 2}, {3.7, 2}, 
 {3.8, 2}, {3.9, 2}, {4., 4}} *)

or, with Interval:

InverseQuantile[list_, x_] := 
 Interval[
  {
   LengthWhile[list // Sort, # < x &]/Length[list], 
   LengthWhile[list // Sort, # <= x &]/Length[list]
  }
 ]

InverseQuantile[{1, 2, 2, 4}, #] & /@ {0, 0.5, 1, 1.5, 2, 3, 4, 5}
(* {Interval[{0, 0}], Interval[{0, 0}], Interval[{0, 1/4}], Interval[{1/4, 1/4}], 
    Interval[{1/4, 3/4}], Interval[{3/4, 3/4}], Interval[{3/4, 1}], Interval[{1, 1}]} *)

Another approach would be to base InverseQuantile on empirical distributions and their CDF. Two possibilities are:

InverseQuantile[list_, x_] := CDF[SmoothKernelDistribution[list], x]

InverseQuantile[list_, x_] := CDF[HistogramDistribution[list], x]

Table[
   {x, Quantile[{1, 2, 2, 4}, InverseQuantile[{1, 2, 2, 4}, x]]}, 
   {x, 1, 4, 0.1}
 ]

    (* ==> {{1., 1}, {1.1, 1}, {1.2, 1}, {1.3, 1}, {1.4, 1}, {1.5, 1}, {1.6, 2},
            {1.7, 2}, {1.8, 2}, {1.9, 2}, {2., 2}, {2.1, 2}, {2.2, 2}, {2.3, 2},
            {2.4, 2}, {2.5, 2}, {2.6, 2}, {2.7, 2}, {2.8, 2}, {2.9, 2}, {3., 2},
            {3.1, 2}, {3.2, 2}, {3.3, 4}, {3.4, 4}, {3.5, 4}, {3.6, 4}, {3.7, 4}, 
            {3.8, 4}, {3.9, 4}, {4., 4}} *)