# How to encode the character into UTF-8?

As we know the café is caf\xc3\xa9 after encode by UTF8:

>>> 'café'.encode('utf8')


b'caf\xc3\xa9'

But if I use

ToCharacterCode["café", "UTF8"]
ImportString["café", CharacterEncoding -> "UTF8"]


{99, 97, 102, 195, 169}

{99, 97, 102, 233}

This not what I after, how to get the real utf8 code(caf\xc3\xa9)?

• 195 base 10 is c3 base 16. 169 base 10 is a9 base 16. The result seems correct to me? – Carl Lange May 7 '19 at 18:24
• if you want to print the ascii and use hex for others, something like If[# > 127, IntegerString[#, 16], FromCharacterCode[#]] & /@ ToCharacterCode["café", "UTF8"] – egwene sedai May 7 '19 at 18:32
• @CarlLange As your prompt, IntegerString[ToCharacterCode["café", "UTF8"], 16] can convert it. But mma have no easier method convert caf\xc3\xa9 into café mutually? – yode May 7 '19 at 18:32

Your string b'caf\xc3\xa9' is the "real utf8 code" in some other language, not Mathematica. You can see the deconstructed UTF-8 in hexadecimal with:

BaseForm[ToCharacterCode["café", "UTF8"], 16]


If you have a bunch of such strings, understand that Mathematica is an expression rewriting language. This is a rewriting problem. A good way to do this is to convert your string to a more elementary expression and rewrite it.

hexReplace[l_List] := l //. {h___, 92, 120, a_, b_, t___} :>
{h, fromHexCode[a, b], t}


The replacement rule takes a list generated by ToCharacterCode, recognizes sequences starting with {92,120} ("\x"), feeds the following two codes to a function fromHexCode, and reassembles the list with the result inserted. ReplaceRepeated (//.) applies this transformation until it has found and replaced every match.

fromHexCode[a_, b_] := FromDigits[FromCharacterCode[{a, b}], 16]


fromHexCode takes a pair of integer character codes representing a pair of hex digits, and combines them into a single integer.

Now, compose a function that does the whole job:

hx2utf8[s_String] := FromCharacterCode[hexReplace[ToCharacterCode[s]], "UTF8"]

hx2utf8["caf\xc3\xa9"]
(* café *)

• Thanks. since it is belong with other language, then caf\xc3\xa9 cannot convert into "café" as you said? – yode May 7 '19 at 18:35
• See edited answer. – John Doty May 7 '19 at 20:13
• Thanks a lot for your work. – yode May 8 '19 at 5:39