15
$\begingroup$

I would like to know how I can remove accents from a string. For example, how can I transform "string test áéíóú" into "string test aeiou"? I have to normalize some text to make comparisons, and this would be very helpful.

|improve this question
$\endgroup$
  • $\begingroup$ It's worth noting that this is a follow-up question to this one. $\endgroup$ – Sjoerd C. de Vries Nov 11 '12 at 19:33
20
$\begingroup$

To remove accents from a string I use this function:

removeAccent[string_] := Module[{accentMap,l1,l2},
    l1 = Characters["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"];
    l2 = Characters["SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"];
    accentMap = Thread[l1 -> l2];
    StringReplace[string, accentMap]
]

So, if you apply it as removeAccent["string test áéíóú"]you get: "string test aeiou"

Update

Now in version 10.1 we have the native function: RemoveDiacritics

RemoveDiacritics["string test áéíóú"] you get "string test aeiou"

Timing comparison using the new RepeatedTiming.

RepeatedTiming[removeAccent["string test áéíóú"]]
RepeatedTiming[RemoveDiacritics["string test áéíóú"]]

> 0.000057
> 0.000015

RemoveDiacritics wins!

|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Ð (ð) is not an accented D (d) but the letter eth without any accents. It should not be considered an accented character in this context. $\endgroup$ – Oleksandr R. Apr 4 '13 at 14:04
  • $\begingroup$ Is RemoveDiacritics not yet documented? $\endgroup$ – Karsten 7. Mar 14 '15 at 3:56
  • $\begingroup$ @Karsten7. Almost, see here $\endgroup$ – Murta Mar 14 '15 at 4:14
  • $\begingroup$ You mean in the next version 10.1 ?? $\endgroup$ – SquareOne Mar 14 '15 at 21:56
  • $\begingroup$ @SquareOne yes! $\endgroup$ – Murta Mar 14 '15 at 22:19
10
$\begingroup$

Your own method without extraneous reevaluation:

With[{accentMap =
  Characters /@
    Rule["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ", 
         "SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"] // Thread},

 removeAccent[string_] := StringReplace[string, accentMap]

]
|improve this answer
$\endgroup$
  • 2
    $\begingroup$ Would a Dispatch[] help matters here? $\endgroup$ – J. M. will be back soon♦ Nov 12 '12 at 9:28
  • 1
    $\begingroup$ I don't think so. I have made some tests with Dispatch, and it gets slower. StringReplacemust already do something like it. $\endgroup$ – Murta Nov 12 '12 at 11:38
  • 1
    $\begingroup$ @J.M. I'll have to test it but I believe that because the String operations are handled by a separate library things like Dispatch are not applicable. At least that's what I seem to recall concluding previously. $\endgroup$ – Mr.Wizard♦ Nov 12 '12 at 16:39
4
$\begingroup$

Works nicely:

removeAccent[s_String] := Module[{patt = "(Capital)?([A-Z]{1})([A-Z]\\w*)*", del}, 
 del = Select[Characters[s], StringMatchQ[ToString[FullForm[#]],
                                          RegularExpression[".*\\[" <> patt <> "\\].*"]] &];
 StringReplace[s, Thread[del -> Map[First, StringCases[ToString[FullForm[#]] & /@ del, 
               RegularExpression[patt] :> If["$1" === "", ToLowerCase, Identity]["$2"]]]]]]

Test:

removeAccent["string test áéíóú"]
   "string test aeiou"

removeAccent["Çärîñő Ð Štùrm"]
   "Carino Ð Sturm"
|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.