The problem does not lie with WordBoundary
, it is due to the use of __
.
The string pattern WordBoundary ~~ __ ~~ "man"
will find a word boundary properly, but then will match a following sequence of characters up to "man"
without restriction -- including characters that lie on word boundaries themselves. To exclude that possibility, we should restrict the characters between WordBoundary
and "man"
to be word characters:
StringCases[mystring, WordBoundary ~~ WordCharacter.. ~~ "man"]
(* {"Superman", "Spiderman", "Batman"} *)
Some Fiddly Details...
Consider the following:
mystring2 = "I like Superman, Spiderman and Batman. But adamantly not 3man!";
StringCases[mystring2, WordBoundary ~~ WordCharacter.. ~~ "man"]
(* { "Superman", "Spiderman", "Batman", "adaman", "3man"} *)
"3man"
was picked up properly, but we probably don't want to see "adaman"
in the list. This is easily corrected:
StringCases[mystring2, WordBoundary ~~ WordCharacter.. ~~"man" ~~ WordBoundary]
(* { "Superman", "Spiderman", "Batman", "3man"} *)
... and a Bug
Now, consider this:
mystring3 = "Remember Underscore_man!";
StringCases[mystring3, WordBoundary ~~ WordCharacter.. ~~"man" ~~ WordBoundary]
(* {} *)
Strange... why didn't the legendary Underscore_man
appear in the result? We find the answer in the documentation for WordCharacter: it represents a letter or digit character. Fair enough. But, wait...
StringCases["Dash-man & Underscore_man", WordBoundary ~~ Shortest@__ ~~ WordBoundary]
(* { "Dash", "-", "man", " & ", "Underscore_man" } *)
I thought that only letters and digits were word characters? But apparently an underscore does not create a word boundary even though it is not a word character. How can this be?
Let's take a look at the regular expressions generated for WordCharacter
and WordBoundary
:
StringPattern`PatternConvert[WordCharacter ~~ WordBoundary] // First
(* (?ms)[[:alnum:]]\b *)
We can see that [[:alnum:]]
is being used for WordCharacter
, in conformance to the statement in the documentation. However, we see that \b
is being used for WordBoundary
. The PCRE documentation tells us that \b
represents adjacent word and non-word characters, and that word characters are letters, digits or underscore.
This means that in Mathematica, there is a mismatch between the definition of WordCharacter
and WordBoundary
. A small bug, to be sure, but beware.