Combine SortBy with ReverseSortBy

Question

Suppose I have a dataset, which I want to sort according to two criteria. However, I want the first criteria to be sorted ascending, and the second descending. This would require to somehow combine SortBy and ReverseSortBy in a single call.

Here is an example:

 dataset = Dataset[{<|"a" -> 1, "b" -> "x", "c" -> {1}|>,
   <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>,
   <|"a" -> 3, "b" -> "z", "c" -> {3}|>,
   <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>,
   <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>,
   <|"a" -> 6, "b" -> "z", "c" -> {}|>}]

Now say I want to sort by "b", in ascending order (x > y > z), and whenever "b" is tied, I want to use "a" as the 2nd criteria, but in descending order (6 > ... > 1). This seem simple, but somehow I can't get my head around it.

If what I wanted was both in ascending, I could simply do

dataset[SortBy[{#b, #a}&]]

The issue is when I want to mix ascending and descending. And I am particularly interested in doing so for non-numeric values.

Thank you in advance for any feedback.

Answer 1

A general way to achieve what you want to do seems to be missing.

Here is a very short solution to sort whatever type of elements according to whatever order you need. It uses the SortBy form with a third argument which is the function used to compare the elements to be sorted.

In your simple case, just define the following ordering function and that's all:

order[rev[x_], rev[y_]] := -Order[x, y];
order[x_, y_] := Order[x, y];

Then, just wrap with rev the arguments you want to be sorted in reverse order.

Examples:

Given your dataset:

SortBy[dataset, {rev[#b] &, #a &}, order]

For the Titanic example used by @kglr :

SortBy[dataset, {rev[#survived] &, #sex &, rev[#class] &, rev[#age] &}, order]

Actually, this hack allows to mix any ordering you want, for example let's add the definition :

order[strlen[x_], strlen[y_]] := If[StringLength[x] <= StringLength[y], 1, -1]

Then for example :

SortBy[dataset, {rev[#survived] &, strlen[#sex] &, rev[#class] &, rev[#age] &}, order]

Answer 2

You can use a list of functions with SortBy (functions that appear later in the list are used to break ties):

dataset[SortBy[{#b &, -#a &}]]

dataset[ReverseSortBy[{#b&, -#a&}]]

dataset[ReverseSortBy[{#b&, #a&}]]

Update: For datasets with non-numeric values we can construct a numeric function using Union (which sorts by the canonical order depending on data type) and PositionIndex for each key:

ClearAll[sBy]
sBy[d_][key_] := Normal[d[Map[PositionIndex@*Union]@*Transpose][key]]

Using a subsample from "Titanic":

SeedRandom[1];
ds = RandomSample[ExampleData[{"Dataset", "Titanic"}], 20] ;

use sBy to create 4 numeric functions:

{fclass, fage, fsex, fsurvived} = sBy[ds] /@ Normal[Keys[ds[1]]];

Check that using {-fsurvived[#survived]&} with SortBy and ReverseSortBy[{#survived&}] gives the same result:

labels = {"ds", "ds[ReverseSortBy[{#survived&}]]", 
  "ds[SortBy[{-fsurvived[#survived]&}]]"};
Grid[{Labeled[ToExpression@#, #, Top] & /@ labels}] 

Sort descending in "survived", ascending in "sex", descending in "class" and descending in "age":

ds[SortBy[{
   -fsurvived[#survived] &, 
   fsex[#sex] &, 
   -fclass[#class] &,
   - #age &}]] 

Answer 3

Here's an approach to convert arbitrary expressions into a form that is sorted "in reverse" (full code at the end):

(* a list of arbitary expressions *)
list = {"a", "b", f[1, 2], f[1]};
Sort@list
(* {"a", "b", f[1], f[1, 2]} *)

(* create a list of wrapped items that will be sorted in reverse *)
list2 = ReverseSorted /@ list
(* {ReverseSorted["a"], ReverseSorted["b"], ReverseSorted[f[1, 2]], ReverseSorted[f[1]]} *)

Sort@list2
(* {ReverseSorted[f[1, 2]], ReverseSorted[f[1]], ReverseSorted["b"], ReverseSorted["a"]} *)

(* convert back *)
Normal[Sort@list2, ReverseSorted]
(* {f[1, 2], f[1], "b", "a"} *)

(* use SortBy to avoid the need for back-conversion *)
SortBy[list, ReverseSorted]
(* {f[1, 2], f[1], "b", "a"} *)

(* example from the question *)
dataset[SortBy[{#b, ReverseSorted@#a} &]]

General strategy

This method works by converting any expression into a canonical form that allows us to control its sorting behavior.

• We wrap all expressions in a ReverseSorted wrapper
  • The wrapper remembers the original expression in the second argument (for back-conversion)
  • The first argument contains a processed form of the wrapped expression that ensures reverse-sorting
• To actually get the reverse-sorting, we look at the canonical ordering for Mathematica, and figure out a way to reverse each rule:
  • The general order is numbers, strings, symbols, expressions
    • -> We create lists of the form {4, number}, {3, string}, etc. This way, the order is controlled by the type of the expression, but since e.g. 4>3, strings will come before numbers, etc,
  • Numbers are sorted by their real part, then imaginary part
    • -> Simply take the negative of the number
  • Strings are sorted lexicographically, with special rules for character ordering
    • -> We generate a list of all characters, sort it, and then replace the first character with the last, the second with the second-to-last, etc. To avoid the length of the string from mattering, we create a nested list from the characters of the string, so {"a", "b", "c"} becomes {"a", {"b", {"c", M[]}}}. The M[] ensures that shorter strings are sorted after longer ones (since M[] comes after List[])
  • Symbols are sorted by their names, then contexts
    • -> We take the name of the symbols and apply the same procedure as for strings (contexts are currently ignored, see also notes in next section)
  • Expressions are sorted by their head, then length, and then arguments
    • -> We create a list {reverseHead, -length, {reverseArgs}}, where the reversing strategy is recursively applied to the head and each argument. (The nesting of the lists ensures that the ordering does not depend on the number of arguments in an uncontrolled way)

Advantages & Limitations

• Compared to the approach by @kglr, this approach does not need "global" knowledge, i.e. you don't need to know the other expressions to be sorted
• Use of this method is very easy, simply wrap any expression (or part of an expression) in ReverseSorted if you want to flip the ordering
• The current version will not work properly for symbols with different contexts, this would have to be implemented properly
• The current version will probably not work properly for some atomic expressions, e.g. Image or Graph, since they are not handled explicitly by the code
• The current version might not work for arbitrary strings if the characters are not in the list CharacterRange[0, 1114111] (which should include any character represented by a single character code, but I'm not sure whether there could be any other characters)

Code

Attributes[ReverseSorted] = {HoldAll};
ReverseSorted[expr_] :=
 ReverseSorted[reverseExpr@expr // Evaluate, expr]
MakeBoxes[full : ReverseSorted[_, expr_], frm : StandardForm] :=
 With[
  {boxes = MakeBoxes[ReverseSorted@expr, frm]},
  InterpretationBox[boxes, full]
  ]
Normal[expr_, ReverseSorted] ^:=
 expr /. ReverseSorted[_, e_] :> e

reverseCharacters =
  ReplaceAll@Dispatch@Thread[# -> SortBy[#, FromCharacterCode]] &@Range[0, 1114111];
toNestedList[{a_, rest___}] :=
 {a, toNestedList@{rest}}
toNestedList[{}] :=
 M[]
reverseString[str_] :=
 toNestedList@reverseCharacters@ToCharacterCode@str

Attributes[reverseExpr] = {HoldFirst};
reverseExpr[n_?NumberQ] :=
 {4, -n}
reverseExpr[s_String] :=
 {3, reverseString@s}
reverseExpr[s : HoldPattern@Except[Symbol[___], _Symbol]] :=
 {2, reverseString@SymbolName@Unevaluated@s}
reverseExpr[h_[args___]] :=
 {1, {reverseExpr[h], -Length@Unevaluated@{args}, reverseExpr /@ Unevaluated@{args}}}