16
$\begingroup$

Bug introduced in 11.2 or earlier and persists through V13.0

[CASE:4026293] confirmed


Let's create 10 asynchronous requests which just check if the site is there.

Do[
  URLSubmit[
    "www.wolfram.com"
  , HandlerFunctionsKeys -> {"StatusCode"}
  , HandlerFunctions -> <|"TaskFinished" -> Print|>
  ]
, {10}
]

Now, depending of Jupiter's position in the sky I get 1 to 7 {} results instead of a valid association:

enter image description here

What do I do wrong?

It happens when each url is different too.

$\endgroup$
7
  • 2
    $\begingroup$ I don't think it is a timeout issue. I used Fiddler to see what was going on and it registered 10 identical request and 10 identical responses. $\endgroup$ Mar 7 '18 at 20:41
  • 3
    $\begingroup$ Mathematica desperately needs promises :) $\endgroup$ Mar 7 '18 at 21:46
  • 9
    $\begingroup$ @LeonidShifrin Isn't already much of Mathematica a promise? There is documentation which promises something, but the promise may or may not be kept. It's a function of time, kind of asynchronous ... (version x.y.z may work in the future some time ) $\endgroup$ Mar 7 '18 at 22:28
  • 1
    $\begingroup$ Do[Pause[.137]; URLSubmit["www.wolfram.com", HandlerFunctionsKeys -> {"StatusCode"}, HandlerFunctions -> <|"TaskFinished" -> Print|>], {10}] works for me ... $\endgroup$ Mar 7 '18 at 22:33
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – Kuba
    Mar 8 '18 at 6:37
10
$\begingroup$

Bug confirmed:

[...] The source of the issue unknown to us at this point; however, it seems that all the request are submitted, yet tasks finishes prematurely, before getting the status code. [...]

So before it gets fixed we can use URLFetchAsynchronous:

statusHandler[obj_, "statuscode", {statusCode_}] := Print@<|"StatusCode" -> statusCode|>;

Do[
 URLFetchAsynchronous["www.wolfram.com", statusHandler],
 {10}
]
$\endgroup$
8
$\begingroup$

This is indeed a racing condition due to TaskObject is created from AsynchronousTaskObject with default value to AutoRemove which is True.

The task is removed manually in 'finished' handler (which fails in some cases due to task is already auto removed). In this case task event data is empty so no status code.

To verify this:

Needs["GeneralUtilities`"];
PrintDefinitions @ URLSubmit

find the lines

spec = <|...|>;
res = AsyncToTaskObject@spec;

add AutoRemove -> False to the spec and press Shift-Enter (you should see 'patched' in the status line).

After that URLSumbit works flawlessly for me.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.