15
$\begingroup$

I would like to know how I can remove accents from a string. For example, how can I transform "string test áéíóú" into "string test aeiou"? I have to normalize some text to make comparisons, and this would be very helpful.

Improve this question
$\endgroup$
1
  • $\begingroup$ It's worth noting that this is a follow-up question to this one. $\endgroup$
    – Sjoerd C. de Vries
    Nov 11, 2012 at 19:33

3 Answers 3

Reset to default
21
$\begingroup$

To remove accents from a string I use this function:

removeAccent[string_] := Module[{accentMap,l1,l2},
    l1 = Characters["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"];
    l2 = Characters["SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"];
    accentMap = Thread[l1 -> l2];
    StringReplace[string, accentMap]
]

So, if you apply it as removeAccent["string test áéíóú"]you get: "string test aeiou"

Update

Now in version 10.1 we have the native function: RemoveDiacritics

RemoveDiacritics["string test áéíóú"] you get "string test aeiou"

Timing comparison using the new RepeatedTiming.

RepeatedTiming[removeAccent["string test áéíóú"]]
RepeatedTiming[RemoveDiacritics["string test áéíóú"]]

> 0.000057
> 0.000015

RemoveDiacritics wins!

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ Ð (ð) is not an accented D (d) but the letter eth without any accents. It should not be considered an accented character in this context. $\endgroup$
    – Oleksandr R.
    Apr 4, 2013 at 14:04
  • $\begingroup$ Is RemoveDiacritics not yet documented? $\endgroup$
    – Karsten 7.
    Mar 14, 2015 at 3:56
  • $\begingroup$ @Karsten7. Almost, see here $\endgroup$
    – Murta
    Mar 14, 2015 at 4:14
  • $\begingroup$ You mean in the next version 10.1 ?? $\endgroup$
    – SquareOne
    Mar 14, 2015 at 21:56
  • $\begingroup$ @SquareOne yes! $\endgroup$
    – Murta
    Mar 14, 2015 at 22:19
10
$\begingroup$

Your own method without extraneous reevaluation:

With[{accentMap =
  Characters /@
    Rule["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ", 
         "SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"] // Thread},

 removeAccent[string_] := StringReplace[string, accentMap]

]
Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Would a Dispatch[] help matters here? $\endgroup$
    – J. M.'s slightly less busy ♦
    Nov 12, 2012 at 9:28
  • 1
    $\begingroup$ I don't think so. I have made some tests with Dispatch, and it gets slower. StringReplacemust already do something like it. $\endgroup$
    – Murta
    Nov 12, 2012 at 11:38
  • 1
    $\begingroup$ @J.M. I'll have to test it but I believe that because the String operations are handled by a separate library things like Dispatch are not applicable. At least that's what I seem to recall concluding previously. $\endgroup$
    – Mr.Wizard
    Nov 12, 2012 at 16:39
4
$\begingroup$

Works nicely:

removeAccent[s_String] := Module[{patt = "(Capital)?([A-Z]{1})([A-Z]\\w*)*", del}, 
 del = Select[Characters[s], StringMatchQ[ToString[FullForm[#]],
                                          RegularExpression[".*\\[" <> patt <> "\\].*"]] &];
 StringReplace[s, Thread[del -> Map[First, StringCases[ToString[FullForm[#]] & /@ del, 
               RegularExpression[patt] :> If["$1" === "", ToLowerCase, Identity]["$2"]]]]]]

Test:

removeAccent["string test áéíóú"]
   "string test aeiou"

removeAccent["Çärîñő Ð Štùrm"]
   "Carino Ð Sturm"
Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.