3
$\begingroup$

I am new to both Mathematica and API's. I am trying to access a dictionary of player-ids to names from the website datdota.com, but some of them have chinese characters.

relevant code: URLExecute["http://datdota.com/api/players/101586543"]

When I run this I get the error "cannot import data as JSON format".

I tried to get around it with URLExecute["http://datdota.com/api/players/101586543", CharacterEncoding -> "Unicode"]

but i got the same error.

Is there a different function I need to use? How can I access this data?

Here's a link to what I'm trying to access: http://datdota.com/api/players/101586543

$\endgroup$
6
  • $\begingroup$ This works fine for me. What version are you using? $\endgroup$ Mar 27, 2018 at 3:54
  • $\begingroup$ Import["http://datdota.com/api/players/101586543", "JSON"] evaluates fine on my end (11.2 on Linux). $\endgroup$ Mar 27, 2018 at 3:57
  • $\begingroup$ @ItaiSeggev pre 11.3 I guess. $\endgroup$
    – Kuba
    Mar 27, 2018 at 7:09
  • $\begingroup$ ImportString[FromCharacterCode[URLRead["http://datdota.com/api/players/101586543", "BodyBytes"]], "JSON"] worked on v11.2 Windows. Stems from [mathematica.stackexchange.com/questions/154245/… also URLExecute["http://datdota.com/api/players/101586543", "Text", CharacterEncoding -> "Unicode"] return an encoded result but difficult to return to JSON that can be parsed. $\endgroup$
    – Hans
    Mar 27, 2018 at 7:12
  • $\begingroup$ @ItaiSeggev I'm using 11.1. on a Mac $\endgroup$ Mar 27, 2018 at 14:08

2 Answers 2

5
$\begingroup$

Oh what a mess:

    ImportString[
 FromCharacterCode[
  ToCharacterCode[
   URLExecute["http://datdota.com/api/players/101586543",  "Text"], 
   "UTF8"]], "JSON"]
(*{"data" -> {"nickname" -> "跳刀跳刀丶", "steam_id" -> "101586543"}}*)

V11.2 Windows

In order to answer poster's questions: "Is there a different function I need to use? How can I access this data?" As @J. M. commented

Import["http://datdota.com/api/players/101586543", "JSON"]

should work. Nevertheless, this particular hic-cup you ran into seems to be a known issue in WL when using URLExcecute. The above workaround solution is there just in case you also need to work with API with more parameter passing.

$\endgroup$
2
  • $\begingroup$ The problem with "Text" may be that it is supposed to infer encoding etc and it is not defined well. "String" should be more stable as kind of raw bytes string but "String" changed in 11.3 while "Text" didn't XD. Daily fun. $\endgroup$
    – Kuba
    Mar 27, 2018 at 8:04
  • $\begingroup$ @Kuba "String" works. Also using the encoding "Custom" in the From and To ..Encode seems to work. I tried many other available encodings such as "ShiftJIS", "UTF-8" over "UTF8"some worked, some did not. $\endgroup$
    – Hans
    Mar 27, 2018 at 17:36
3
$\begingroup$

It could be related to all this 'content-type' mess: Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails, but there are no messages issued before failure so I'm not so sure.

I think the issue was known because:

  • V10.4 returns $Failed and Import::fmterr: Cannot import data as JSON format. >>

  • V11.2 returns Failure with `ImportFailure tag. No messages.

  • V11.3 returns correct association

For pre 11.3 use either Import or linked route through "BodyBytes":

ImportString[ FromCharacterCode @ URLRead[ 
  "http://datdota.com/api/players/101586543", "BodyBytes"]
, "RawJSON"
]
<|"data" -> <|"nickname" -> "跳刀跳刀丶", "steam_id" -> "101586543"|>|>
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.