# String pattern search and replacement — it works, but produces error messages only the first time it is called

I defined a function in Mathematica to process a pre-generated string. More specifically, I'd like to process some parts of C codes generated from MMA. Basically what my function does is to expand the pow function in C, so it involves finding a string pattern and replacing it. I know using MMA is not the best way to solve this problem, but unfortunately the language I'm most familiar with is MMA...Anyway, here's my function:

ExpandPowInCode[code_] :=
StringReplace[code, Shortest["pow(" ~~ x__ ~~ "," ~~ y__ ~~ ")"] ->
If[StringMatchQ[x, ___ ~~ "+" ~~ __] ||
StringMatchQ[x, ___ ~~ "-" ~~ __],
Nest[StringJoin["(" <> ToString[x] <> ")", "*" ~~ #] &,
"(" <> ToString[x] <> ")", ToExpression[y, InputForm] - 1],
Nest[StringJoin[x, "*" ~~ #] &, x, ToExpression[y, InputForm] - 1]]]


It works as I expected, namely expanding , for example, "pow(x,2)" into "x*x". The If condition will help when dealing with plus/minus sign in the first argument of pow. For example,

ExpandPowInCode["pow(a x+b,3)"]


"(a x+b)*(a x+b)*(a x+b)"

However, at the first time I executed ExpandPowInCode, I always got the following error messages:

StringMatchQ::strse: String or list of strings expected at position 1 in StringMatchQ[x,___~~+~~__]. >>

StringMatchQ::strse: String or list of strings expected at position 1 in StringMatchQ[x,___~~-~~__]. >>

Although ExpandPowInCode still gave me the correct result, the appearance of error messages was really annoying...Moreover, when I executed the function again later, the error messages no longer appeared as if everything were correct. Has anyone ever met this odd situation?

-
You might consider replacing -> with :>... –  Guess who it is. Jun 7 '13 at 15:17
@0x4A4D you always give me great suggestions! It works without error message now, but I still wonder why in my original definition with -> the error messages only appeared once? Shouldn't they pop out every time I execute it? –  Leo Fang Jun 7 '13 at 15:21
Not sure why. Anyway, something for your consideration: ExpandPowInCode[code_String] := StringReplace[code, RegularExpression["pow$(.+?),(\\s*)(\\d+)$"] :> ToString[Row[ConstantArray["(" <> "$1" <> ")", ToExpression["$3"]], "*"]]] –  Guess who it is. Jun 7 '13 at 15:55
I can't understand quite what is happening, but by running TraceScan[Print, ExpandPowInCode["pow(a x+b,3)"]] twice, you can see at what point in the execution the difference is occuring. It is, not surprisingly, diverging some time before the error occurs, just after Shortest[pow(~~x__~~,~~y__~~)]. –  Jonathan Shock Jun 7 '13 at 17:06
btw, consider that it is still unknown why the error messages popped up only once, I hope this question could remain open so that I may learn the reason (or pitfall of manipulating strings) later from someone else's answer. Thanks. –  Leo Fang Jun 7 '13 at 18:58

The simple answer is that the problem (and message) can be avoided by correctly using RuleDelayed (:>) rather than Rule (->) in your string replacement, preventing the evaluation of StringMatchQ before the string is substituted into it.

I believe the lack of a message on the second execution is explained by the caching or limiting of infinite evaluation that Mathematica uses. This is a subject that I do not have a good understanding of therefore I cannot provide any true explanation, only observations; take any declarative statements as opinion only.

Consider the Trace output of this:

f = {head[3, "+"], Pi} &;

f[] // TracePrint


f[]
f
List
3
+
[Pi]

And a second evaluation:

f[]
f

At least in some circumstances an expression that is comprised solely of sub-expressions that do not evaluate appears to be marked as not needing evaluation if it is seen again.

This behavior can be seen in a Trace of your question code, or in this condensed example:

x=.
g[s_String] := StringMatchQ[x, "+"]

g["string"];
g["string"] // TracePrint


g[string]

g

string

StringMatchQ[x,+]

Notice that the second time g["string"] is evaluated the expression StringMatchQ[x,+] is never actually evaluated; its components do not appear in the Trace. Compare this behavior to the same code with a subexpression that will always evaluate:

h[s_String] := StringMatchQ[2 + 2, "+"]

h["string"];
h["string"] // TracePrint


h[string]
h
string
StringMatchQ[2+2,+]
StringMatchQ
2+2
Plus
2
2
4
+
StringMatchQ[4,+]
Message[StringMatchQ::strse,1,StringMatchQ[4,+]]
(* remainder trimmed *)

The h function issues the message every time it is run because, due to the presence of 2 + 2, it is never marked as not needing evaluation.

-
I too suspected caching and I like your reasoning, so it gets my vote. However, I don't quite agree with the fact that StringMatchQ[x, ...] is a pattern that doesn't need evaluation when x is a symbol (and not a local var). For instance, it needs to know if x has changed state, and indeed it does issue the warning again if you set x = 1. I don't know if there are other ways it is made aware of this fact — perhaps through the FE? If that's the case, then this decision might be made at parsing. However, I'm even more of a novice in this area than you are, so I could well be wrong. –  The Toad Jun 9 '13 at 13:21
@rm-rf I observed the same thing. I don't mean to suggest that it never needs evaluation, but rather it is (apparently) marked as not presently needing further evaluation. I suppose the symbols upon which the expression depends are also noted and if one of these changes it triggers reevaluation on next appearance. –  Mr.Wizard Jun 9 '13 at 13:38
I believe my hunch that it is done at the parsing stage is correct. Try this: f[x_String] := (z = x /. y_String -> StringMatchQ[y, "+"]; z) g[x_String] := (Clear@y; z = x /. y_String -> StringMatchQ[y, "+"]; y = 1; z) with f["+"] and g["+"] (twice) –  The Toad Jun 9 '13 at 16:17
Mr. Wizard's explanation is correct. StringMatchQ[x, ...] (when x is a symbol with no value) evaluates to itself, and doesn't trigger any side effects, so it is marked as not needing reevaluation. It is also marked as depending on x, so if you change the value of x, or do Update[x], then the expression will need reevaluation, and you will see the message again. –  Todd Gayley Jun 9 '13 at 17:12
@LeoFang Remember that the StringMatchQ[x, ___ ~~ "+" ~~ __] sub-expression doesn't depend on the input string until after a string value is substituted for x. The warning message is issued before that point, when the expression is evaluated for symbolic x. Once that evaluation is done once, it never needs to be done again, no matter what argument you pass the function. –  Todd Gayley Jun 10 '13 at 20:49