I would like to know how I can remove accents from a string. For example, how can I transform "string test áéíóú"
into "string test aeiou"
?
I have to normalize some text to make comparisons, and this would be very helpful.
-
$\begingroup$ It's worth noting that this is a follow-up question to this one. $\endgroup$– Sjoerd C. de VriesNov 11, 2012 at 19:33
3 Answers
To remove accents from a string I use this function:
removeAccent[string_] := Module[{accentMap,l1,l2},
l1 = Characters["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"];
l2 = Characters["SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"];
accentMap = Thread[l1 -> l2];
StringReplace[string, accentMap]
]
So, if you apply it as removeAccent["string test áéíóú"]
you get: "string test aeiou"
Update
Now in version 10.1 we have the native function: RemoveDiacritics
RemoveDiacritics["string test áéíóú"]
you get "string test aeiou"
Timing comparison using the new RepeatedTiming
.
RepeatedTiming[removeAccent["string test áéíóú"]]
RepeatedTiming[RemoveDiacritics["string test áéíóú"]]
> 0.000057 > 0.000015
RemoveDiacritics
wins!
-
1$\begingroup$ Ð (ð) is not an accented D (d) but the letter eth without any accents. It should not be considered an accented character in this context. $\endgroup$ Apr 4, 2013 at 14:04
-
-
-
-
Your own method without extraneous reevaluation:
With[{accentMap =
Characters /@
Rule["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ",
"SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"] // Thread},
removeAccent[string_] := StringReplace[string, accentMap]
]
-
2$\begingroup$ Would a
Dispatch[]
help matters here? $\endgroup$ Nov 12, 2012 at 9:28 -
1$\begingroup$ I don't think so. I have made some tests with Dispatch, and it gets slower.
StringReplace
must already do something like it. $\endgroup$– MurtaNov 12, 2012 at 11:38 -
1$\begingroup$ @J.M. I'll have to test it but I believe that because the String operations are handled by a separate library things like
Dispatch
are not applicable. At least that's what I seem to recall concluding previously. $\endgroup$ Nov 12, 2012 at 16:39
Works nicely:
removeAccent[s_String] := Module[{patt = "(Capital)?([A-Z]{1})([A-Z]\\w*)*", del},
del = Select[Characters[s], StringMatchQ[ToString[FullForm[#]],
RegularExpression[".*\\[" <> patt <> "\\].*"]] &];
StringReplace[s, Thread[del -> Map[First, StringCases[ToString[FullForm[#]] & /@ del,
RegularExpression[patt] :> If["$1" === "", ToLowerCase, Identity]["$2"]]]]]]
Test:
removeAccent["string test áéíóú"]
"string test aeiou"
removeAccent["Çärîñő Ð Štùrm"]
"Carino Ð Sturm"