9
$\begingroup$

I am interested in counting a number of uppercase letters in the text. I do it in the following manner.

 text = "This place is situated in Agra, Uttar Pradesh, India on State 
 Highway 62 around 70 km from Agra city and around 55 km from Etawah. 
 Its geographical coordinates are 26 52' 12" North, 78
 \[Degree] 35' 51" East. Three rivers, Yamuna, Chambal and Utangan 
 irrigate its land and separate from the states Madhya Pradesh and 
   Rajsthan"

 res = LetterCounts[StringDelete[text, PunctuationCharacter]]
 Total[Boole[UpperCaseQ /@ Keys[res]]]

Any suggestion how to speed up this calculation

$\endgroup$
  • $\begingroup$ I voted to close, but I am only 75% pro-closing. The answer of @rhermans shows that this can be done fast with StringCount. $\endgroup$ – Anton Antonov Jun 30 '18 at 19:41
  • $\begingroup$ ok, fair enough. $\endgroup$ – Anton Antonov Jul 1 '18 at 10:13
  • $\begingroup$ Please take a look at the @C.E solutions'. This solution is an order of magnitude faster, and what more important in my opinion it is elegant solutions that not use StringCount. $\endgroup$ – Kiril Danilchenko Jul 1 '18 at 17:25
13
$\begingroup$

Using rhermans' testtext, I would propose this:

HammingDistance[testtext, ToLowerCase[testtext]] // RepeatedTiming

{0.00025, 185}

(Update: ) Here is another one:

uppercase = Alternatives @@ CharacterRange["A", "Z"];
StringCount[testtext, uppercase] // RepeatedTiming

{0.00066, 185}

Using Length@StringCases instead of StringCount takes about the same amount of time, 0.00068 in my experiment.

Also, Mr. Wizard mentioned using regular expressions, which can be made fast in the same way (which is not surprising because string patterns are internally converted to regular expressions):

regex = RegularExpression@First@StringPattern`PatternConvert[uppercase];
StringCount[testtext, regex] // RepeatedTiming

{0.00067, 185}

However, on my computer this is only as fast as using the more appropriate regular expression suggested by Mr. Wizard, it is neither better nor worse.

We can compare this to rhermans' approach:

RepeatedTiming[StringCount[testtext, _?UpperCaseQ]]

{0.00336, 185}

Both the HammingDistance approach and the StringCases approach are an order of magnitude faster.

$\endgroup$
  • 2
    $\begingroup$ Very elegant solution $\endgroup$ – Kiril Danilchenko Jul 1 '18 at 5:51
  • $\begingroup$ Regarding your second method I think it is worth noting that CharacterRange["A", "Z"], equivalent to RE [A-Z] is not the same as UpperCaseQ as there are many upper case letters according to the latter that do not fall in that basic set of 26. This is why I used \p{Lu} instead. ToLowerCase handles the extended set properly, so your first solution is robust. $\endgroup$ – Mr.Wizard Jul 2 '18 at 20:22
  • $\begingroup$ @Mr.Wizard That is why I said "the more appropriate regular expression suggested by Mr. Wizard". However, I don't see the speed of this method varying much with a few more characters in the pattern. If you need to add ten or twenty characters to make it robust, then the performance should not necessarily deteriorate that much. $\endgroup$ – C. E. Jul 2 '18 at 21:31
8
$\begingroup$

It seems to me you are counting the number of different letters that apear as uppercase and not the total number of uppercase characters present. Here I use the UN Declaration of Human Rights without PunctuationCharacter as sample text. I use RepeatedTiming to show the average time in seconds used to compute, together with the result obtained.

Test data

testtext = StringDelete[
   ExampleData[{"Text", "UNHumanRightsEnglish"}]
   , PunctuationCharacter];

Your approach

RepeatedTiming[
 Total[Boole[UpperCaseQ /@ Keys[LetterCounts[testtext]]]]
 ]
(* {0.0564, 21} *)

What I think you intended

RepeatedTiming[
 Total@KeySelect[LetterCounts[testtext], UpperCaseQ]
 ]
(* {0.059, 185} *)

My approach

RepeatedTiming[
 StringCount[testtext, _?UpperCaseQ]
 ]
(* {0.0032, 185} *)

@C.E. offers the best answer so far, using HammingDistance. Using a similar approach but with EditDistance it's the slowest of all.

BarChart[
 First /@ {
    RepeatedTiming[(* "HammingDistance" @C.E *)
     HammingDistance[testtext, ToLowerCase[testtext]]
     ],
    RepeatedTiming[(* "StringCount Alternatives CharacterRange" @C.E *)


     StringCount[testtext, Alternatives @@ CharacterRange["A", "Z"]]
     ],
    RepeatedTiming[(* "StringCount RegularExpression" @Mr.Wizard *)
     StringCount[testtext, RegularExpression["\\p{Lu}"]]
     ],
    RepeatedTiming[ (* "StringCount UpperCaseQ" @rhermans *)
     StringCount[testtext, _?UpperCaseQ]
     ],
    RepeatedTiming[(* "Length@StringCases" *)
     Length@StringCases[testtext, _?UpperCaseQ]
     ],
    RepeatedTiming[(* "Total KeySelect LetterCounts" @
     KirilDanilchenko *)
     Total@KeySelect[LetterCounts[testtext], UpperCaseQ]
     ],
    RepeatedTiming[(* "EditDistance" @rhermans *)
     EditDistance[testtext, ToLowerCase[testtext]]
     ]
    } 10^6
 , ChartLabels -> {
   "HammingDistance\n@C.E",
   "StringCount\nAlternatives\nCharacterRange\n@C.E",
   "StringCount\nRegularExpression\n@Mr.Wizard ",
   "StringCount\nUpperCaseQ\n@rhermans",
   "Length@StringCases",
   "Total\nKeySelect\nLetterCounts\n@KirilDanilchenko",
   "EditDistance\n@rhermans"
   }
 , PlotTheme -> "Scientific"
 , AspectRatio -> 1/2
 , ImageSize -> 600
 , FrameLabel -> {None, "Time \[Mu]s"}
 , BarSpacing -> Large
 , ScalingFunctions -> "Log"
 ]

Mathematica graphics

$\endgroup$
  • $\begingroup$ ooh, I see I made a mistake. I want to count the total number of uppercase letters :( $\endgroup$ – Kiril Danilchenko Jul 1 '18 at 4:12
  • $\begingroup$ I added a version using StringCases that is fast to my answer. $\endgroup$ – C. E. Jul 1 '18 at 10:01
  • 2
    $\begingroup$ @C.E. Plot is now updated. $\endgroup$ – rhermans Jul 1 '18 at 14:54
4
$\begingroup$

On my system (v10.1 under Windows) RegularExpression is not quite as fast as HammingDistance but arguably it is more flexible, and much faster than UpperCaseQ

text = ExampleData[{"Text", "DeclarationOfIndependence"}];

StringCount[text, RegularExpression["\\p{Lu}"]]  // RepeatedTiming
{0.000420, 342}

cf.

StringCount[text, _?UpperCaseQ]                  // RepeatedTiming
HammingDistance[text, ToLowerCase[text]]         // RepeatedTiming
{0.00573, 342}
{0.000300, 342}
$\endgroup$
  • $\begingroup$ I always like RegularExpression in cases like these :) It's in some sense the right tool for the job, even if my re knowledge is rust. $\endgroup$ – b3m2a1 Jul 1 '18 at 6:20
  • $\begingroup$ @b3m2a1 That's the downside for me too; I had to reference stackoverflow.com/a/4052294 for this. $\endgroup$ – Mr.Wizard Jul 1 '18 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.