# How to ImportString "JSON" with Unicode characters? [duplicate]

## Update

The short answer is that when getting results back from an API into an HTTPResponse object the "Body" property is a post-decoded form of the result. ImportString with "JSON" expects the encoded form of the result. Therefore you should use the "BodyBytes" property with FromCharacterCode.

With res a HTTPResponse object with the result, then

ImportString[FromCharacterCode@res["BodyBytes"], "JSON"]


brings in the result without issue.

URLExecute can also be used to directly return the imported JSON but you don't get a nice HTTPResponse object to check the status of your result.

With req a HTTPRequest object, then

URLExecute[req, {}, "JSON"]


brings in the result without issue.

## OP

I have many large JSON results from a web API that contain Unicode characters. ImportString with "JSON" seems unable to import these. As a minimal example try,

ImportString["{\"id\":1,\"text\":\"ñía\"}", "JSON"]


Import::utf8expcontinuation: Input is not a valid UTF8 byte sequence. 237 is expected to be a continuation byte.

Import::jsonhintposition: An error occurred at line 1:19

I have tried adding option CharacterEncoding -> "UTF8" and get the same result.

Finally I tried

# -> ImportString["{\"id\":1,\"text\":\"ñía\"}", "JSON",
CharacterEncoding -> #] & /@ $CharacterEncodings  Every encoding failed. How do I import Unicode JSON? I have many such results to import. I am using Win10 Mma 11.3 • Round tripping to and from ByteArray works on your small example for some encodings: Cases[Quiet[# -> ImportString[ByteArrayToString[StringToByteArray["{\"id\":1,\"text\":\"ñía\"}"], #], "JSON"]] & /@$CharacterEncodings, _[_, _List]] Mar 21, 2019 at 23:26
• @ChipHurst A few of them do work with the round trip. Any ideas why hoops must be jumped through to get what should be a basic import. Bug, perhaps? Mar 22, 2019 at 0:04
• It feels like a bug, but I don’t have enough background to know for sure. Mar 22, 2019 at 0:05
• Where does this string come from? If from URLRead Body then it is not the way to go and you have to import bytes > FromCharacterCode > ImportString. For explanation see the first section in mathematica.stackexchange.com/q/154245/5478
– Kuba
Mar 22, 2019 at 0:21
• @Edmund yes if URLExecute can be used then go with it. Sometimes one needs to use URLRead when a more fine grained control is needed.
– Kuba
Mar 22, 2019 at 13:43