Pattern Matching, FullForm and DisplayForm

Question

Has anybody an explanation for the following or is it a bug? The display form and the input form do not give the same result.

I include the raw display form here. Simply copy it to Mathematica to inspect it. The two following two pattern match:

MatchQ[
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(2\)], 
TagBox[
RowBox[{"(", 
RowBox[{"1", ",", "0"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2], 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(_\)], 
TagBox[
RowBox[{"(", 
RowBox[{"_", ",", "_"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2]]

MatchQ[
 Derivative[0, 1][Subscript[u, 2]][x1, x2] 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(2\)], 
TagBox[
RowBox[{"(", 
RowBox[{"0", ",", "1"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2], 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(_\)], 
TagBox[
RowBox[{"(", 
RowBox[{"_", ",", "_"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2]   
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(_\)], 
TagBox[
RowBox[{"(", 
RowBox[{"_", ",", "_"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2]]

However if I change the first derivative from Fullform to Displayform, that is: Derivative[0,1][u2] to u2^(0,1), the pattern does no more match:

MatchQ[
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(2\)], 
TagBox[
RowBox[{"(", 
RowBox[{"0", ",", "1"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2] 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(2\)], 
TagBox[
RowBox[{"(", 
RowBox[{"1", ",", "0"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2], 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(_\)], 
TagBox[
RowBox[{"(", 
RowBox[{"_", ",", "_"}], ")"}],
Derivative],
MultilineFunction->None]\) [x1, x2] 
\!\(\*SuperscriptBox[
SubscriptBox[\(u\), \(_\)], 
TagBox[
RowBox[{"(", 
RowBox[{"_", ",", "_"}], ")"}],
Derivative],
MultilineFunction->None]\)[x1, x2]]

Answer 1

Summary

The exhibited behaviour is correct. In the last example, the expression under test is an inert expression of the form Derivative[...] * Derivative[...]. But the pattern evaluates to the form Derivative[...]^2. These two forms do not match. The fix is to use HoldPattern to prevent the pattern from evaluating. When doing this kind of work, it probably would not hurt to hold the test expression Unevaluated as well.

Details

To make it easier to see what is happening, we will work with InputForm throughout.

First Example

The first example is straight-forward:

MatchQ[
  Derivative[1,0][x1,x2]
, Derivative[_,_][x1,x2]
]

(* True *)

Second Example

The second example seems to be straight-forward as well:

MatchQ[
  Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[0,1][Subscript[u,2]][x1,x2]
, Derivative[_,_][Subscript[u,_]][x1,x2]*Derivative[_,_][Subscript[u,_]][x1,x2]
]

(* True *)

... but the route to this result might be a little surprising. MatchQ does not hold its arguments, so both the test expression and the pattern are evaluated before they are compared:

Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[0,1][Subscript[u,2]][x1,x2]
(* Derivative[0,1][Subscript[u,2]][x1,x2]^2 *)

Derivative[_,_][Subscript[u,_]][x1,x2]*Derivative[_,_][Subscript[u,_]][x1,x2]
(* Derivative[_,_][Subscript[u,_]][x1,x2]^2 *)

Lucky for us, these evaluated forms are still structural matches:

MatchQ[
  Derivative[0,1][Subscript[u,2]][x1,x2]^2
, Derivative[_,_][Subscript[u,_]][x1,x2]^2
]

(* True *)

Third Example

In the third example, we are not so lucky:

MatchQ[
  Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2]
, Derivative[_,_][Subscript[u,_]][x1,x2]*Derivative[_,_][Subscript[u,_]][x1,x2]
]

(* False *)

Because the expression under test is a product of derivatives that are taken with respect to different variables, it does not evaluate to the form ... ^ 2. But the pattern does:

Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2]
(* Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2] *)

Derivative[_,_][Subscript[u,_]][x1, x2]*Derivative[_,_][Subscript[u,_]][x1,x2]    
(* Derivative[_,_][Subscript[u,_]][x1,x2]^2 *)

So the forms that we are actually matching are manifestly different:

MatchQ[
  Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2]
, Derivative[_,_][Subscript[u,_]][x1,x2]^2
]

(* False *)

Fix

The fix is to ensure that the pattern is not evaluated prior to use. We can use HoldPattern:

MatchQ[
              Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2]
, HoldPattern[Derivative[_,_][Subscript[u,_]][x1,x2]*Derivative[_,_][Subscript[u,_]][x1,x2]]
]

(* True *)

Depending upon the application, it may be desirable to prevent the test expression from evaluating as well. If so, Unevaluated might be a convenient choice:

MatchQ[
  Unevaluated[Derivative[0,1][Subscript[u,2]][x1,x2]*Derivative[1,0][Subscript[u,2]][x1,x2]]
, HoldPattern[Derivative[_,_][Subscript[u,_]][x1,x2]*Derivative[_,_][Subscript[u,_]][x1,x2]]
]

(* True *)

Other methods to prevent evaluation are available, but they are beyond the scope of this response.

Display Forms

All of these results shown here using InputForm will be the same if we work in display forms as well. For example: