10
$\begingroup$

Bug introduced in 12.0


The following HTTP request fails to submit the body under version 12, but works in Mathematica 11:

(* set up "diagnostic" request target *)
url = CloudDeploy[Delayed[HTTPRequestData[], "WL"]];
(* build and execute request *)
req = HTTPRequest[
   url, <|
    Method -> "GET", 
    "Body" -> "test"
    |>
   ];
URLRead[req, "Body"]

Output in version 11 (truncated)

<|"AbsoluteDomain" -> "http://www.wolframcloud.com:80/", 
 "AbsolutePath" -> "http://www.wolframcloud.com:80/obj/8cea...",
 "Body" -> "test", "DispatchPathString" -> "/", 
 "Domain" -> "www.wolframcloud.com", "FormRules" -> {}, 
 "Headers" -> {"host" -> "www.wolframcloud.com", "accept" -> "*/*", 
   "accept-encoding" -> "deflate, gzip", 
   "content-type" -> "text/plain;charset=utf-8", 
   "user-agent" -> "Wolfram HTTPClient 11.3"},
 "Method" -> "GET", 
 "MultipartElements" -> None, "Parameters" -> {}, 
 "Path" -> {"", "obj", "8cea..."}, 
 "PathString" -> "/obj/8cea...", 
 "Port" -> "80", "Query" -> {}, "QueryString" -> "", "Scheme" -> "http"|>

Output in version 12 (notice the empty body tag)

<|"AbsoluteDomain" -> "http://www.wolframcloud.com:80/", 
 "AbsolutePath" -> "http://www.wolframcloud.com:80/obj/70ee...", 
 "Body" -> "",
  "DispatchPathString" -> "/", "Domain" -> "www.wolframcloud.com", 
 "FormRules" -> {}, 
 "Headers" -> {"host" -> "www.wolframcloud.com", "accept" -> "*/*", 
   "accept-encoding" -> "deflate, gzip", 
   "content-type" -> "text/plain;charset=utf-8", 
   "user-agent" -> "Wolfram HTTPClient 12."},
 "Method" -> "GET",
 "MultipartElements" -> None, "Parameters" -> {}, 
 "Path" -> {"", "obj", "70ee..."}, 
 "PathString" -> "/obj/70ee...", 
 "Port" -> "80", "Query" -> {}, "QueryString" -> "", "Scheme" -> "http"|>

Has there been a change in version 12 that is changing the behavior of URLRead[]?

Original question

The following code queries Elasticsearch and has always worked fine with Mathematica 11 but no longer works with version 12.

URLRead[<|
    "Scheme" -> "https",
    "Domain" -> "elastic-hot-10.internal.lynch.cc",
    "Port" -> 9200,
    "Path" -> "trade.exec.tally/_search",
    "Method" -> "GET",
    "Headers" -> {
        "Authorization" -> "Basic " <> esNodeCreds,
        "Content-Type" -> "application/json"},
    "Body" -> ExportString[{
        "query" -> {
            "regexp" -> {
                "dataspec" -> "gdax_1509525900000_1509529499999.*"
            }
        }
    }, "JSON"]
|>, VerifySecurityCertificates -> False]["Body"]

With version 11 the query is executed and correctly returns the records. But for version 12 the query is basically empty and so it returns ALL records from the index.

If I look at the query logs for Elasticsearch I see that the log for the code from version 11 correctly shows the query:

{
  "type": "index_search_slowlog",
  "timestamp": "2019-10-02T21:25:08,631+0000",
  "level": "DEBUG",
  "component": "i.s.s.fetch",
  "cluster.name": "elasticsearch_zero",
  "node.name": "elastic-hot-10.internal.lynch.cc",
  "cluster.uuid": "uobOmkQ6SKOzYcTavaP9LA",
  "node.id": "nEaT_A-JRUOnjJnz2aG1ng",
  "message": "[trade.exec.tally-000002][0] took[395.1micros], took_millis[0], total_hits[1409 hits], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[6], source[{\"query\":{\"regexp\":{\"dataspec\":{\"value\":\"gdax_1509525900000_1509529499999.*\",\"flags_value\":65535,\"max_determinized_states\":10000,\"boost\":1.0}}}}], id[], "
}

But for the code from version 12 you can see the the source for the query is empty:

{
  "type": "index_search_slowlog",
  "timestamp": "2019-10-02T21:26:07,476+0000",
  "level": "DEBUG",
  "component": "i.s.s.query",
  "cluster.name": "elasticsearch_zero",
  "node.name": "elastic-hot-10.internal.lynch.cc",
  "cluster.uuid": "uobOmkQ6SKOzYcTavaP9LA",
  "node.id": "nEaT_A-JRUOnjJnz2aG1ng",
  "message": "[trade.exec.tally-000002][0] took[54.7micros], took_millis[0], total_hits[4504 hits], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[6], source[{}], id[], "
}

Has there been a change in version 12 that is changing the behavior of URLRead[]?

$\endgroup$
  • 2
    $\begingroup$ I have added a self-contained minimal working example to the question, so people can more easily play around with it. Feel free to revert the edit if I misunderstood or missed something $\endgroup$ – Lukas Lang Oct 2 '19 at 22:27
  • $\begingroup$ @LukasLang Perfect. I was planning on coming up with a more minimal example but it's been a long day and I just phoned it in. Thank you! $\endgroup$ – Sean Lynch Oct 2 '19 at 22:46
  • $\begingroup$ Is this a bug? Or just a change that we can adjust for? If it's a bug, what's the next steps? If it's just a change, what has to be changed? $\endgroup$ – Sean Lynch Oct 3 '19 at 0:31
4
$\begingroup$

According to Wolfram support this is a bug, the team knows about it, and it will be fixed in a future version.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.