Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I want to perform a simple analysis of some Greek text: collect the different words used and count their frequency. It seems that some of the built-in commands do not work well with Greek letters. For instance, with

words = StringCases["α β1 rpr other", WordCharacter ..]

the output is

{"1", "rpr", "other"}

How do I get Mathematica to recognize other kinds of words?

share|improve this question
1  
FWIW, you might consider something like StringCases["α β1 rpr0 other", RegularExpression["(\\w|[\[CapitalAlpha]-ω])+"]]. A bit cumbersome, but there ya go... with the caveat that this can't handle characters with tonos; you'll have to modify the regex as needed. –  J. M. Apr 26 '13 at 11:26
    
It'd good to be aware of this: forums.wolfram.com/mathgroup/archive/2009/Jul/msg00398.html –  Szabolcs Apr 26 '13 at 14:43
add comment

2 Answers

up vote 4 down vote accepted

I have partial success in splitting Greek text:

greek = ExampleData[{"Text", "HomerOdysseyGreek"}]
Style[StringTake[greek, 100], FontFamily -> "Times"]
Style[StringSplit[StringTake[greek, 100], " "], FontFamily -> "Times"]

greek

share|improve this answer
    
StringSplit is essentially all I need, and it works great. It correctly identifies words in the Greek. –  GregH Apr 26 '13 at 12:18
add comment

I'd say this is either a programming or a documentation bug.

The documentation for WordCharacter says:

WordCharacter matches any character for which either LetterQ or DigitQ yields True. »

Well, WordCharacter clearly doesn't consider alpha a letter:

StringMatchQ["α", WordCharacter]

False

but LetterQ does:

LetterQ["α"]

True

A workaround:

theGreeks=Alternatives@@Select[CharacterRange["\[CapitalAlpha]", "\[Omega]"], LetterQ];

StringCases["α β1 rpr other", (WordCharacter | theGreeks) ..]

{"α", "β1", "rpr", "other"}

You may want to enlarge the set of characters included in "theGreeks". Now only, the basic Greek characters are in there.

share|improve this answer
1  
FWIW, both StringMatchQ["α", RegularExpression["[[:word:]]"]] and StringMatchQ["α", RegularExpression["\\w"]] also return False. –  J. M. Apr 26 '13 at 11:19
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.