Background
Taking an example from Decoding GZIP encoded Body, BodyBytes (ByteArray) and BodyBytesArray from URLRead:
URLRead[
"https://api.stackexchange.com/2.2/info?site=mathematica", "Headers"
]
{ ... , "content-type" -> "application/json; charset=utf-8" , "content-encoding" -> "gzip" , ... }
Import @ "https://api.stackexchange.com/2.2/info?site=mathematica"
returns string json which can be later put to ImportString
.
Problem
URLRead
though throws a bunch of decoding errors suggesting it ignores "gzip"
spec and goes directly to charset
specified in content-type
.
URLRead[
"https://api.stackexchange.com/2.2/info?site=mathematica", "Body"
]
Workaround
is already shown in linked topic:
URLRead[
"https://api.stackexchange.com/2.2/info?site=mathematica"
, "BodyBytes"
] // FromCharacterCode // ImportString[#, {"gzip", "RawJSON"}] &
Question
Should that be the case? Is it a bug or am I missing the purpose of URLRead
Body
?
URLFetch
behaves the same so I'm surprised it wasn't asked before.
URLFetch[
"https://api.stackexchange.com/2.2/info?site=mathematica"
, "Content"
]
Related
Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails
Import
actually does not need the"content-encoding" -> "gzip"
header for recognizinggzip
compressed data. You can check it withfile=URLDownload["https://api.stackexchange.com/2.2/info?site=mathematica"];Import@file
. The createdfile
is a binarygzip
-compressed file andImport
recognizes the compression method from the first few bytes of the file, the HTTP"content-encoding"
header isn't necessary at all. $\endgroup$Import
worked, from the quoted question.. BTW, "As of Version 11,URLFetch
has been superseded byURLRead
andURLExecute
." $\endgroup$gzip
is of crucial importance for such function asURLRead
. As I wrote above, I would expect it to recognize gzip even without the explicit"content-encoding" -> "gzip"
. Probably the latter can be checked using thefile:
protocol. $\endgroup$