question about using Heads->True in Cases command

Question

I was trying to find the order of highest derivative of expression. Using solution given in detect-highest-order-of-derivative-in-expression

I just do not understand why when removing Heads->True then Cases do not work.

Here is example. I want to find the order of all derivatives (then I can use Max to find the largest)

ClearAll[y,x,n]
expr=y''[x]+y'[x]+x+y[x] Sin[x];
Cases[expr,Derivative[n_]->n,Infinity,Heads->True]
(*  {1,2} *)

but

 Cases[expr,Derivative[n_]->n,Infinity]
 (* {} *)

From help ref/Cases it just says that this option will just include the Head in the result:

But I do not see how and why this will made Cases find the pattern. I do not even see the Head returned back in the result. May the above documentation should say more about this.

What is the rule of the thumb as to when to use Heads->True in Cases? Since this seems to be very critical in making it work in this example, and I do not want to just try each time to see if it works or not.

Answer 1

When Cases searches an expression, it assumes by default Heads -> False. This means that expression heads are not searched for matches. With Heads -> True, heads are searched.

By means of the visualization helper function frameMatches (defined at the bottom of this post), we can explore the implications of this option.

First, let's look at the full form of the expression in question:

expr // FullForm

Now, let's highlight all of the subexpressions (i.e. anything that matches _), but ignoring heads:

frameMatches[expr, _, Infinity, Heads -> False]

This is the full set of possible matches for an unrestricted pattern. Note that Derivative[1], Derivative[2], Derivative[1][y], Derivative[2][y] and the other heads are not in this set -- even though the expressions that contain them are. So it is no surprise that it we try to match the pattern Derivative[_] we come up empty:

frameMatches[expr, Derivative[_], Infinity, Heads -> False]

Cases[expr, Derivative[_], Infinity, Heads -> False]
(* {} *)

Now, let's include heads when matching. Again, with the unrestricted pattern _:

frameMatches[expr, _, Infinity, Heads -> True]

This time around, Derivative[1] and Derivative[2] are included in the search. And the search even descends into the heads of those expressions, to include Derivative.

This is why we now get the results we want using the desired pattern:

frameMatches[expr, Derivative[_], Infinity, Heads -> True]

Cases[expr, Derivative[_], Infinity, Heads -> True]
(* {Derivative[1], Derivative[2]} *)

Matching Heads with Heads -> False

Incidentally, the option Heads -> True is not the only way to match heads within an expression. We can write explicit patterns that will extract the heads from matched non-head parts. For example:

Cases[expr, (d : Derivative[_])[_][_] :> d, Infinity]
(* {Derivative[1], Derivative[2]} *)

Is All This Documented?

Yes, but as usual it is terse. The documentation for Heads says:

Heads is an option for functions which use level specifications that specifies whether heads of expressions should be included.

Heads->True treats heads just like other elements of expressions for the purpose of levels.

Heads->False never includes heads as part of any level of an expression.

---

frameMatches

caveat: this function works for the examples at hand, but is not intended to be fully general

frameMatches[expr_, patt_, args___] :=
  Replace[expr // FullForm, e:patt :> Framed[e], args] // expandFrames

expandFrames[e_] :=
  MakeBoxes[e] //
  Replace[#, RowBox[{"Framed", "[", x_, "]"}] :> FrameBox[x], Infinity]& //
  Replace[RowBox[{FrameBox["FullForm", ___],"[", x_, "]"}] :> x] //
  RawBoxes