9
$\begingroup$

I am interested in counting a number of uppercase letters in the text. I do it in the following manner.

 text = "This place is situated in Agra, Uttar Pradesh, India on State 
 Highway 62 around 70 km from Agra city and around 55 km from Etawah. 
 Its geographical coordinates are 26 52' 12" North, 78
 \[Degree] 35' 51" East. Three rivers, Yamuna, Chambal and Utangan 
 irrigate its land and separate from the states Madhya Pradesh and 
   Rajsthan"

 res = LetterCounts[StringDelete[text, PunctuationCharacter]]
 Total[Boole[UpperCaseQ /@ Keys[res]]]

Any suggestion how to speed up this calculation

$\endgroup$
3
  • $\begingroup$ I voted to close, but I am only 75% pro-closing. The answer of @rhermans shows that this can be done fast with StringCount. $\endgroup$ Jun 30, 2018 at 19:41
  • $\begingroup$ ok, fair enough. $\endgroup$ Jul 1, 2018 at 10:13
  • $\begingroup$ Please take a look at the @C.E solutions'. This solution is an order of magnitude faster, and what more important in my opinion it is elegant solutions that not use StringCount. $\endgroup$ Jul 1, 2018 at 17:25

3 Answers 3

13
$\begingroup$

Using rhermans' testtext, I would propose this:

HammingDistance[testtext, ToLowerCase[testtext]] // RepeatedTiming

{0.00025, 185}

(Update: ) Here is another one:

uppercase = Alternatives @@ CharacterRange["A", "Z"];
StringCount[testtext, uppercase] // RepeatedTiming

{0.00066, 185}

Using Length@StringCases instead of StringCount takes about the same amount of time, 0.00068 in my experiment.

Also, Mr. Wizard mentioned using regular expressions, which can be made fast in the same way (which is not surprising because string patterns are internally converted to regular expressions):

regex = RegularExpression@First@StringPattern`PatternConvert[uppercase];
StringCount[testtext, regex] // RepeatedTiming

{0.00067, 185}

However, on my computer this is only as fast as using the more appropriate regular expression suggested by Mr. Wizard, it is neither better nor worse.

We can compare this to rhermans' approach:

RepeatedTiming[StringCount[testtext, _?UpperCaseQ]]

{0.00336, 185}

Both the HammingDistance approach and the StringCases approach are an order of magnitude faster.

$\endgroup$
3
  • 2
    $\begingroup$ Very elegant solution $\endgroup$ Jul 1, 2018 at 5:51
  • $\begingroup$ Regarding your second method I think it is worth noting that CharacterRange["A", "Z"], equivalent to RE [A-Z] is not the same as UpperCaseQ as there are many upper case letters according to the latter that do not fall in that basic set of 26. This is why I used \p{Lu} instead. ToLowerCase handles the extended set properly, so your first solution is robust. $\endgroup$
    – Mr.Wizard
    Jul 2, 2018 at 20:22
  • $\begingroup$ @Mr.Wizard That is why I said "the more appropriate regular expression suggested by Mr. Wizard". However, I don't see the speed of this method varying much with a few more characters in the pattern. If you need to add ten or twenty characters to make it robust, then the performance should not necessarily deteriorate that much. $\endgroup$
    – C. E.
    Jul 2, 2018 at 21:31
8
$\begingroup$

It seems to me you are counting the number of different letters that apear as uppercase and not the total number of uppercase characters present. Here I use the UN Declaration of Human Rights without PunctuationCharacter as sample text. I use RepeatedTiming to show the average time in seconds used to compute, together with the result obtained.

Test data

testtext = StringDelete[
   ExampleData[{"Text", "UNHumanRightsEnglish"}]
   , PunctuationCharacter];

Your approach

RepeatedTiming[
 Total[Boole[UpperCaseQ /@ Keys[LetterCounts[testtext]]]]
 ]
(* {0.0564, 21} *)

What I think you intended

RepeatedTiming[
 Total@KeySelect[LetterCounts[testtext], UpperCaseQ]
 ]
(* {0.059, 185} *)

My approach

RepeatedTiming[
 StringCount[testtext, _?UpperCaseQ]
 ]
(* {0.0032, 185} *)

@C.E. offers the best answer so far, using HammingDistance. Using a similar approach but with EditDistance it's the slowest of all.

BarChart[
 First /@ {
    RepeatedTiming[(* "HammingDistance" @C.E *)
     HammingDistance[testtext, ToLowerCase[testtext]]
     ],
    RepeatedTiming[(* "StringCount Alternatives CharacterRange" @C.E *)


     StringCount[testtext, Alternatives @@ CharacterRange["A", "Z"]]
     ],
    RepeatedTiming[(* "StringCount RegularExpression" @Mr.Wizard *)
     StringCount[testtext, RegularExpression["\\p{Lu}"]]
     ],
    RepeatedTiming[ (* "StringCount UpperCaseQ" @rhermans *)
     StringCount[testtext, _?UpperCaseQ]
     ],
    RepeatedTiming[(* "Length@StringCases" *)
     Length@StringCases[testtext, _?UpperCaseQ]
     ],
    RepeatedTiming[(* "Total KeySelect LetterCounts" @
     KirilDanilchenko *)
     Total@KeySelect[LetterCounts[testtext], UpperCaseQ]
     ],
    RepeatedTiming[(* "EditDistance" @rhermans *)
     EditDistance[testtext, ToLowerCase[testtext]]
     ]
    } 10^6
 , ChartLabels -> {
   "HammingDistance\[email protected]",
   "StringCount\nAlternatives\nCharacterRange\[email protected]",
   "StringCount\nRegularExpression\[email protected] ",
   "StringCount\nUpperCaseQ\n@rhermans",
   "Length@StringCases",
   "Total\nKeySelect\nLetterCounts\n@KirilDanilchenko",
   "EditDistance\n@rhermans"
   }
 , PlotTheme -> "Scientific"
 , AspectRatio -> 1/2
 , ImageSize -> 600
 , FrameLabel -> {None, "Time \[Mu]s"}
 , BarSpacing -> Large
 , ScalingFunctions -> "Log"
 ]

Mathematica graphics

$\endgroup$
3
  • $\begingroup$ ooh, I see I made a mistake. I want to count the total number of uppercase letters :( $\endgroup$ Jul 1, 2018 at 4:12
  • $\begingroup$ I added a version using StringCases that is fast to my answer. $\endgroup$
    – C. E.
    Jul 1, 2018 at 10:01
  • 2
    $\begingroup$ @C.E. Plot is now updated. $\endgroup$
    – rhermans
    Jul 1, 2018 at 14:54
4
$\begingroup$

On my system (v10.1 under Windows) RegularExpression is not quite as fast as HammingDistance but arguably it is more flexible, and much faster than UpperCaseQ

text = ExampleData[{"Text", "DeclarationOfIndependence"}];

StringCount[text, RegularExpression["\\p{Lu}"]]  // RepeatedTiming
{0.000420, 342}

cf.

StringCount[text, _?UpperCaseQ]                  // RepeatedTiming
HammingDistance[text, ToLowerCase[text]]         // RepeatedTiming
{0.00573, 342}
{0.000300, 342}
$\endgroup$
2
  • $\begingroup$ I always like RegularExpression in cases like these :) It's in some sense the right tool for the job, even if my re knowledge is rust. $\endgroup$
    – b3m2a1
    Jul 1, 2018 at 6:20
  • $\begingroup$ @b3m2a1 That's the downside for me too; I had to reference stackoverflow.com/a/4052294 for this. $\endgroup$
    – Mr.Wizard
    Jul 1, 2018 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.