2
$\begingroup$

I searched for a WL implementation of the Python Pandas qcut function but could not find one. Anyone aware of an implementation?

For the moment I just need quartiles and the assignment of values in each quartile to specified labels. I came up with the following two ways to implement it.

SeedRandom[314];
data = RandomReal[50, 20];
labels = {a, b, c, d};

Method 1

Use BinLists to bin the data according to the quartile ranges. Create an association that maps values to labels.

quartileLabel1[data_, labels_] := Module[{bins, binned, map},
  bins = {-Infinity, Quartiles[data], Infinity} // Flatten;
  binned = BinLists[data, {bins}];
  map = Merge[AssociationThread[First[#], Last[#]] & /@ Transpose[{binned, labels}], First];
  map /@ data
  ]

quartileLabel1[data, labels]
(* {d, c, b, a, a, b, a, c, d, d, b, a, b, c, d, d, c, b, a, c} *)

Method 2

Use the quartiles to generate arguments for Which, call it from a function and map that function over the data.

quartileLabel2[data_, labels_] := Module[{quartiles, binLabels, mapper},
  quartiles = Quartiles[data];
  binLabels = Transpose[{quartiles, Most[labels]}];
  mapper[x_] = 
   Which[Evaluate[
     Sequence @@ 
      Flatten[{x <= First[#], Last[#]} & /@ binLabels // Append[{True, Last[labels]}]]]];
  mapper /@ data
  ]

quartileLabel2[data, labels]
(* {d, c, b, a, a, b, a, c, d, d, b, a, b, c, d, d, c, b, a, c} *)

Any ideas for a better / simpler way to implement this?

$\endgroup$
  • 1
    $\begingroup$ data /. MapThread[Alternatives @@ # -> #2 &, {binList, labels}] where binList is the BinLists result is a different way to write the first solution (but may not be as performant). I think BinLists is the way to go here, that way you're using the appropriate algorithm for binning the values. $\endgroup$ – C. E. Jan 23 at 3:31
  • $\begingroup$ @C.E. I did consider using replacement rules but, as you pointed out, I was concerned about performance. The lists I am working with have ~10K distinct elements and that is a lot of rules. However, your suggestion did provide a cleaner way to generate the map. map = Merge[MapThread[ AssociationThread[# -> #2] &, {binned, labels}], First]. Thanks. $\endgroup$ – Rohit Namjoshi Jan 23 at 22:23
2
$\begingroup$
ClearAll[f0]
f0 = Module[{q = Quartiles[#], d= #, l = #2}, 
   l[[1 + Total[Developer`ToPackedArray[UnitStep[# - q & /@ d]], {2}]]]] &;

This gives the same output as quartileLabel1 and quartileLabel2:

f0[data, labels] == quartileLabel1[data, labels] == quartileLabel2[data, labels]

True

Timing comparisons with quartileLabel1and quartileLabel2:

SeedRandom[314];
dt = RandomReal[50, #] & /@ {1000, 10000, 100000, 1000000};

Table[First[RepeatedTiming[rf0[j] = f0[dt[[j]], labels]]], {j, 1, 4}]

{0.00032, 0.0023, 0.025, 0.262}

Table[First[RepeatedTiming[rql1[j] = quartileLabel1[dt[[j]], labels]]], {j, 1,  4}]

{0.0023, 0.023, 0.4, 5.3}

Table[First[RepeatedTiming[rql2[j] = quartileLabel2[dt[[j]], labels]]], {j, 1,  4}]

{0.0033, 0.02, 0.144, 1.404}

And @@ (rf0[#] == rql1[#] == rql2[#] & /@ Range[4])

True

$\endgroup$
  • $\begingroup$ Not in my wildest dreams would I have come up with this solution. Very impressive. Thank you! $\endgroup$ – Rohit Namjoshi Jan 23 at 22:15
  • $\begingroup$ @RohitNamjoshi, my pleasure. Thank you for the accept and kind words. $\endgroup$ – kglr Jan 23 at 22:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.