13
$\begingroup$

In a comment to related question Oleksandr R. made an interesting remark:

... because string patterns are converted into regular expressions and matched using PCRE, they are subject to some strange and perhaps unexpected limitations versus ordinary patterns. One can easily be trapped by the generality of expression patterns and write a string pattern that can't actually be expressed as a regular expression.

I think it would be very useful both from practical and educational points of view to have a list of types of string patterns which can't be converted into regular expressions. Periodically when I write a string pattern it doesn't work for unobvious reason and having a set of non-convertible constructs (especially with short explanations why conversion is impossible) would be very enlightening.

$\endgroup$
  • $\begingroup$ Except["some text"]. see here and your answer. $\endgroup$ – andre314 Sep 2 '16 at 15:26
3
$\begingroup$

Two examples are given under the "Implementation Details" section of the official tutorial "Working with String Patterns" (thanks to WReach for the pointer):

Because PCRE currently does not support preset character classes with characters beyond character code 255, the word and letter character classes (such as WordCharacter and LetterCharacter) only include character codes in the Unicode range 0–255. Thus LetterCharacter and _?LetterQ do not give equivalent results beyond character code 255.

Because of a similar PCRE restriction, case-insensitive matching (for example, with IgnoreCase->True) will only apply to letters in the Unicode range 0–127 (that is, the normal English letters "a"–"z" and "A"–"Z").

The above statements aren't completely correct because PCRE currently supports Unicode, and starting from Mathematica version 10 WordCharacter and LetterCharacter also support characters beyond character code 255. Unfortunately it is still not the case for the IgnoreCase -> True option.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.