3
$\begingroup$

I am new to both Mathematica and API's. I am trying to access a dictionary of player-ids to names from the website datdota.com, but some of them have chinese characters.

relevant code: URLExecute["http://datdota.com/api/players/101586543"]

When I run this I get the error "cannot import data as JSON format".

I tried to get around it with URLExecute["http://datdota.com/api/players/101586543", CharacterEncoding -> "Unicode"]

but i got the same error.

Is there a different function I need to use? How can I access this data?

Here's a link to what I'm trying to access: http://datdota.com/api/players/101586543

$\endgroup$
  • $\begingroup$ This works fine for me. What version are you using? $\endgroup$ – Itai Seggev Mar 27 '18 at 3:54
  • $\begingroup$ Import["http://datdota.com/api/players/101586543", "JSON"] evaluates fine on my end (11.2 on Linux). $\endgroup$ – J. M. will be back soon Mar 27 '18 at 3:57
  • $\begingroup$ @ItaiSeggev pre 11.3 I guess. $\endgroup$ – Kuba Mar 27 '18 at 7:09
  • $\begingroup$ ImportString[FromCharacterCode[URLRead["http://datdota.com/api/players/101586543", "BodyBytes"]], "JSON"] worked on v11.2 Windows. Stems from [mathematica.stackexchange.com/questions/154245/… also URLExecute["http://datdota.com/api/players/101586543", "Text", CharacterEncoding -> "Unicode"] return an encoded result but difficult to return to JSON that can be parsed. $\endgroup$ – Hans Mar 27 '18 at 7:12
  • $\begingroup$ @ItaiSeggev I'm using 11.1. on a Mac $\endgroup$ – Matthew Rivers Mar 27 '18 at 14:08
4
$\begingroup$

Oh what a mess:

    ImportString[
 FromCharacterCode[
  ToCharacterCode[
   URLExecute["http://datdota.com/api/players/101586543",  "Text"], 
   "UTF8"]], "JSON"]
(*{"data" -> {"nickname" -> "跳刀跳刀丶", "steam_id" -> "101586543"}}*)

V11.2 Windows

In order to answer poster's questions: "Is there a different function I need to use? How can I access this data?" As @J. M. commented

Import["http://datdota.com/api/players/101586543", "JSON"]

should work. Nevertheless, this particular hic-cup you ran into seems to be a known issue in WL when using URLExcecute. The above workaround solution is there just in case you also need to work with API with more parameter passing.

$\endgroup$
  • $\begingroup$ The problem with "Text" may be that it is supposed to infer encoding etc and it is not defined well. "String" should be more stable as kind of raw bytes string but "String" changed in 11.3 while "Text" didn't XD. Daily fun. $\endgroup$ – Kuba Mar 27 '18 at 8:04
  • $\begingroup$ @Kuba "String" works. Also using the encoding "Custom" in the From and To ..Encode seems to work. I tried many other available encodings such as "ShiftJIS", "UTF-8" over "UTF8"some worked, some did not. $\endgroup$ – Hans Mar 27 '18 at 17:36
2
$\begingroup$

It could be related to all this 'content-type' mess: Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails, but there are no messages issued before failure so I'm not so sure.

I think the issue was known because:

  • V10.4 returns $Failed and Import::fmterr: Cannot import data as JSON format. >>

  • V11.2 returns Failure with `ImportFailure tag. No messages.

  • V11.3 returns correct association

For pre 11.3 use either Import or linked route through "BodyBytes":

ImportString[ FromCharacterCode @ URLRead[ 
  "http://datdota.com/api/players/101586543", "BodyBytes"]
, "RawJSON"
]
<|"data" -> <|"nickname" -> "跳刀跳刀丶", "steam_id" -> "101586543"|>|>
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.