Background

Taking an example from Decoding GZIP encoded Body, BodyBytes (ByteArray) and BodyBytesArray from URLRead:

URLRead[
]

{ ...
, "content-type" -> "application/json; charset=utf-8"
, "content-encoding" -> "gzip"
, ...
}

Import @ "https://api.stackexchange.com/2.2/info?site=mathematica"


returns string json which can be later put to ImportString.

Problem

URLRead though throws a bunch of decoding errors suggesting it ignores "gzip" spec and goes directly to charset specified in content-type.

URLRead[
"https://api.stackexchange.com/2.2/info?site=mathematica", "Body"
]


Workaround

URLRead[
"https://api.stackexchange.com/2.2/info?site=mathematica"
, "BodyBytes"
] // FromCharacterCode // ImportString[#, {"gzip", "RawJSON"}] &


Question

Should that be the case? Is it a bug or am I missing the purpose of URLRead Body?

URLFetch behaves the same so I'm surprised it wasn't asked before.

URLFetch[
"https://api.stackexchange.com/2.2/info?site=mathematica"
, "Content"
]


Related

Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails

• Note that Import actually does not need the "content-encoding" -> "gzip" header for recognizing gzip compressed data. You can check it with file=URLDownload["https://api.stackexchange.com/2.2/info?site=mathematica"];Import@file. The created file is a binary gzip-compressed file and Import recognizes the compression method from the first few bytes of the file, the HTTP "content-encoding" header isn't necessary at all. Aug 29, 2017 at 14:20
• @AlexeyPopkov good point, so no one cares about content-encoding :)
– Kuba
Aug 29, 2017 at 14:21
• I would like to understand this too (+1). We knew that Import worked, from the quoted question.. BTW, "As of Version 11, URLFetch has been superseded by URLRead and URLExecute." Aug 29, 2017 at 14:23
• @rhermans and Alexey, now it works, does it mean it was a bug?
– Kuba
Oct 17, 2017 at 21:04
• @Kuba I think it was a bug since ability to recognize gzip is of crucial importance for such function as URLRead. As I wrote above, I would expect it to recognize gzip even without the explicit "content-encoding" -> "gzip". Probably the latter can be checked using the file: protocol. Oct 18, 2017 at 9:52

As of V11.2 Content-Encoding specification is respected by URLRead and e.g.
URLRead[