Skip to main content
1 of 7
Wjx
  • 9.7k
  • 1
  • 34
  • 70

How to Speed Pattern Matching up!

#Main Problem

Mr.Wizard poked this question when discussing in another problem.

I will copy the main part of Mr.Wizards problem here:

MatchQ[{1, 2, 3, 4, 5}, {x__?((Echo[##]; False) &), y__}]

Consider this simple Pattern matching, we get the result as:

1

1

1

1

False

The result is correct of course, and there should be four possible pattern, but It seems that though MatchQ already know the element 1 cannot pass the pattern test, it continues to match it for three more times.

It is okay here as evaluation is fast enough, but this will consume a lot of time while matching long lists or dealing with patterns with long evaluation time, such like x__?((Echo[##];Pause@1;False)&)

So the main problem is:

how to speed this process up and aviod repetitive calculation.


##Some more explanation

I think some walk around is needed to get better results:

  1. Different from Condition(/;), PatternTest will check every element in __?test like pattern, not putting everything together. Thus this two pattern matching is different, and they give different results:

     MatchQ[{1, 2, 3}, {__?((Echo@{##}; True) &)}];
    
     MatchQ[{1, 2, 3}, {x__ /; (Echo@{x}; True)}];
    

{1}

{2}

{3}

{1,2,3}

Thus, in most occasions, there'll be no problem of interference while using PatternTest, for example, testing 1 in Sequence[1,2] will usually be the same as testing 1 in Sequence[1,2,3]. While in Condition, interference will be significant.

  1. There do is few examples of interference, take the answer of @Leonid Shifrin under the question I've mentioned at the beginning:

     Module[{flag = False}, ClearAll[test3]; 
       test3[x_] := With[{fl = flag}, If[! flag, flag = True]; Echo@x; fl]];
    
     MatchQ[{-1,2,3,4,5},{x__?test3,y__}];
    

-1

-1

2

But I think these occasions usually won't occur, thus I still want to know are there any method avaliable to tell Mathematica "I know testing result of elements will keep the same in each trial, SKIP those testing!!!"?


##Some Notes

I'm NOT finding a way to store the result of the pattern test result, I'm trying to reduce the testing repetitions.

The latter one will speed up the simple pattern matching process in long lists considerably faster while the first one won't, as the pattern test complete in a flash, but there're simply too much "flashes", thus the speed is low......

Actually the sample match I used is an example, four tests complete in a flash, so storing will not speed it up. But If we can aviod the testing of the latter three 1, the whole process will be 4 times faster!

Wjx
  • 9.7k
  • 1
  • 34
  • 70