6
$\begingroup$

When importing content containing Chinese (CJK) characters, correct results can be obtained by using URLread

URLRead["https://zhuanlan.zhihu.com/p/35359905", "Body"]

screenshot of correct output

But when using URLSubmit, the output is garbled

URLSubmit[
 HTTPRequest["https://zhuanlan.zhihu.com/p/35359905", 
  CharacterEncoding -> "UTF-8"], 
 HandlerFunctions -> <|"BodyReceived" -> ((body = #Body) &)|>]

screenshot: output contains weird characters

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ It seems that the notion of "Body" is not consistent across different functions. URLRead's Body is already decoded with respect to response's headers, here ContentType->text/html; charset=utf-8. For URLSubmit is is raw string, so either use Jean-Pierre's answer or FromCharacterCode[ToCharacterCode@#Body, "UTF8"]. $\endgroup$
    – Kuba
    Commented Jul 3, 2022 at 22:09
  • $\begingroup$ @Kuba I can't run your code well ![jJCDIJ.png](imgtu.com/i/jJCDIJ) $\endgroup$
    – a15355447898a
    Commented Jul 4, 2022 at 3:27

1 Answer 1

Reset to default
8
$\begingroup$

It appears that using BodyByteArray is the preferred approach.

URLSubmit[
 HTTPRequest["https://zhuanlan.zhihu.com/p/35359905", 
  CharacterEncoding -> "UTF-8"], 
 HandlerFunctions -> <|
   "BodyReceived" -> ((body = 
        ByteArrayToString@#[["BodyByteArray"]]) &)|>, 
 HandlerFunctionsKeys -> {"BodyByteArray"}]
Improve this answer
$\endgroup$
1
  • $\begingroup$ It works because ByteArrayToString is "assuming it contains UTF-8 data", in general it could not be the case and decoding should be based on information in response's headers. $\endgroup$
    – Kuba
    Commented Jul 4, 2022 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.