I am trying to extract from a string all words that end with "-man", but the problem is that WordBoundary does not seem to work as it returns whole string ignoring actual word boundaries.

Here is the code:

mystring = "I am a big fan of Superman, Spiderman and Batman.";
StringCases[mystring, WordBoundary ~~ __ ~~ "man"];

4 Answers 4


The problem does not lie with WordBoundary, it is due to the use of __.

The string pattern WordBoundary ~~ __ ~~ "man" will find a word boundary properly, but then will match a following sequence of characters up to "man" without restriction -- including characters that lie on word boundaries themselves. To exclude that possibility, we should restrict the characters between WordBoundary and "man" to be word characters:

StringCases[mystring, WordBoundary ~~ WordCharacter.. ~~ "man"]

(* {"Superman", "Spiderman", "Batman"} *)

Some Fiddly Details...

Consider the following:

mystring2 = "I like Superman, Spiderman and Batman.  But adamantly not 3man!";

StringCases[mystring2, WordBoundary ~~ WordCharacter.. ~~ "man"]

(* { "Superman", "Spiderman", "Batman", "adaman", "3man"} *)

"3man" was picked up properly, but we probably don't want to see "adaman" in the list. This is easily corrected:

StringCases[mystring2, WordBoundary ~~ WordCharacter.. ~~"man" ~~ WordBoundary]

(* { "Superman", "Spiderman", "Batman", "3man"} *)

... and a Bug

Now, consider this:

mystring3 = "Remember Underscore_man!";

StringCases[mystring3, WordBoundary ~~ WordCharacter.. ~~"man" ~~ WordBoundary]

(* {} *)

Strange... why didn't the legendary Underscore_man appear in the result? We find the answer in the documentation for WordCharacter: it represents a letter or digit character. Fair enough. But, wait...

StringCases["Dash-man & Underscore_man", WordBoundary ~~ Shortest@__ ~~ WordBoundary]

(* { "Dash", "-", "man", " & ", "Underscore_man" } *)

I thought that only letters and digits were word characters? But apparently an underscore does not create a word boundary even though it is not a word character. How can this be?

Let's take a look at the regular expressions generated for WordCharacter and WordBoundary:

StringPattern`PatternConvert[WordCharacter ~~ WordBoundary] // First

(* (?ms)[[:alnum:]]\b *)

We can see that [[:alnum:]] is being used for WordCharacter, in conformance to the statement in the documentation. However, we see that \b is being used for WordBoundary. The PCRE documentation tells us that \b represents adjacent word and non-word characters, and that word characters are letters, digits or underscore.

This means that in Mathematica, there is a mismatch between the definition of WordCharacter and WordBoundary. A small bug, to be sure, but beware.

  • $\begingroup$ Finally an answer that said everything I was going to, plus an interesting analysis of WordBoundy. +1 :-) $\endgroup$
    – Mr.Wizard
    Aug 3, 2014 at 19:15
  • 1
    $\begingroup$ @WReach Another small bug: Intersection[{"Batman", "Spiderman", "Superman"}, DictionaryLookup["*man"]] $\endgroup$
    – eldo
    Aug 3, 2014 at 19:30
  • $\begingroup$ @eldo Marvel licenses aren't cheap. :) $\endgroup$
    – WReach
    Aug 3, 2014 at 20:56
  • $\begingroup$ Note, that some non-letter characters are LetterCharacters and are not considered WordBoundary: {StringReplace["aǁa a|a", WordBoundary ~~ "a" ~~ WordBoundary -> "A"], StringMatchQ["ǁ", LetterCharacter]}. $\endgroup$ May 14, 2019 at 10:40

There is a reason to use WordBoundary, but your example sentence doesn't bring it out. Consider

mystring = "I am a demanding fan of Superman, Spiderman and Batman.";

StringCases[mystring, LetterCharacter ... ~~ "man"]
{"deman", "Superman", "Spiderman", "Batman"}

so you might prefer

StringCases[mystring, LetterCharacter ... ~~ "man" ~~ WordBoundary]
{"Superman", "Spiderman", "Batman"}

In your example StringCases works as expected because by default it searches the Longest substring which matches the pattern __ (by default all repeated string patterns are greedy). Apparently the whole string without ending "." matches your pattern:

In[21]:= mystring = "I am a big fan of Superman, Spiderman and Batman.";
substr = StringCases[mystring, WordBoundary ~~ __ ~~ "man"]
StringMatchQ[substr, WordBoundary ~~ __ ~~ "man"]    
Out[22]= {"I am a big fan of Superman, Spiderman and Batman"}

Out[23]= {True}

One way to restrict the pattern for matching only single words ending with "man" is to use LetterCharacter instead of "any character" _:

In[24]:= StringCases[mystring, WordBoundary ~~ LetterCharacter .. ~~ "man" ~~ WordBoundary]
Out[24]= {"Superman", "Spiderman", "Batman"}

Note that switching to lazy version (Shortest) of the repeated pattern does not give the desired result:

In[25]:= StringCases[mystring, Shortest[WordBoundary ~~ __ ~~ "man" ~~ WordBoundary]]
Out[25]= {"I am a big fan of Superman", ", Spiderman", " and Batman"}

To steal examples from the page:

str1 = "I am a big fan of Superman, Spiderman and Batman.";
str2 = "I am a demanding fan of Ironman, Superman and other man-made \
str3 = "I like Superman, Spiderman and Batman.  But adamantly not \
str4 = "Remember Underscore_man!";


f[str_] := Module[{words = Echo@TextWords[str]},
  Select[StringEndsQ[_ ~~ "man"]][words]

f /@ {str1, str2, str3, str4}


{{"Superman", "Spiderman", "Batman"}, {"Ironman", "Superman"}, {"Superman", "Spiderman", "Batman", "3man"}, {"Underscore_man"}}

  • $\begingroup$ Users can remove Echo if debugging is not needed. $\endgroup$
    – Syed
    Mar 3 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.