Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Edit: as noted by Albert Retey the performance difference is only seen when sub expression extraction is performed. If this test is used below the timings are similar:

First@Timing[r1 = StringCases[textBig, se];]
First@Timing[r2 = StringCases[textBig, re];]

According to the documentation:

Any symbolic string pattern is first translated to a regular expression. You can see this translation by using the internal StringPattern`PatternConvert function.

StringPattern`PatternConvert["a" | "" ~~ DigitCharacter ..] // InputForm
{"(?ms)a?\\d+", {}, {}, Hold[None]}

The first element returned is the regular expression, while the rest of the elements have to do with conditions, replacement rules, and named patterns.

The regular expression is then compiled by PCRE, and the compiled version is cached for future use when the same pattern appears again. The translation from symbolic string pattern to regular expression only happens once.

Based on this I would expect a StringExpression and the regular expression produced by PatternConvert to perform similarly, but they do not. Taking an example from this recent question please observe:

se = Shortest["(ICD-9-CM " ~~ code__ ~~ ")"];
re = First @ StringPattern`PatternConvert[se] // RegularExpression
RegularExpression["(?ms)\\(ICD-9-CM (.+?)\\)"]
text1 = "  A  Vitamin D Deficiency (ICD-9-CM 268.9) (ICD-9-CM 268.9) 09/11/2015  01 ";
textBig = StringJoin @ ConstantArray[text1, 1*^6];

First@Timing[r1 = StringCases[textBig, se :> code];]
First@Timing[r2 = StringCases[textBig, re :> "$1"];]

r1 === r2
0.718
1.903
True
  • Why is using the StringExpression more than twice as fast as the RegularExpression?
  • Is there a way to make the RegularExpression matching run just as quickly?
share|improve this question
    
I think you have v7. Just mentioning that it's the same in 9 too. –  Szabolcs May 22 '13 at 15:59
    
The answer might be in StringPattern`PatternConvert[re]... which is not the same as re. –  Szabolcs May 22 '13 at 16:00
1  
@Szabolcs ?: means clustering but not capturing, so you can group regexes within (?:) but doesn't make backreferences as () does. –  Stefan May 22 '13 at 16:26
1  
it seems like that StringExpression does something magical with RegularExpression. Presumably it does compile/cache the pattern –  Stefan May 22 '13 at 17:06
2  
I have no time to investigate that more closely, but I think the difference could well come from how the resubstituting of the matches is implemented. If you try the same thing without such substitutions, then the runtime differences are marginal, e.g.: StringCases[textBig, RegularExpression["(?ms)\\(ICD-9-CM .+?\\)"]] vs. StringCases[textBig, Shortest["(ICD-9-CM " ~~ __ ~~ ")"]] seem to be equally fast (or probably slow when compared to pcregrep :-) –  Albert Retey May 22 '13 at 19:56
show 10 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.