9
$\begingroup$

I am wondering how to most quickly log data during cloud execution. I do not need anything fancy: all I want to do is persist certain information for later analysis, with minimal blocking for the end user. A solution that involves dumping data and then periodically cleaning it up with a ScheduledTask would be acceptable. All of the code will run in the cloud. The Permissions need to be private.

For example, say I want to track which users call certain functions I have deployed.

CloudSymbol is faster in the cloud than in Mathematica, but it is still slow.

CloudSymbol@"list1" = {};
AppendTo[CloudSymbol@"list1", $RequesterWolframUUID] // AbsoluteTiming

{4.070239, $RequesterWolframUUID}

Before we compare timings, briefly note this behavior:

CloudSymbol@"list1"

{$RequesterWolframUUID}

CloudEvaluate[CloudSymbol@"list1"]

{"0e2497dc-9281-48f3-8e84-14b5e2587446"}

CloudSymbol@"list1"

{$RequesterWolframUUID}

Returning to timings,

CloudEvaluate[AppendTo[CloudSymbol@"list1",$RequesterWolframUUID]//AbsoluteTiming]
  //AbsoluteTiming

{2.866184, {1.723062, {0e2497dc-9281-48f3-8e84-14b5e2587446,0e2497dc-9281-48f3-8e84-14b5e2587446}}}

It is worth pointing out that the assignment in the cloud caused $RequesterWolframUUID to be resolved at that time. Compare this result to the behavior above.

CloudSymbol@"list1"

{0e2497dc-9281-48f3-8e84-14b5e2587446,0e2497dc-9281-48f3-8e84-14b5e2587446}

URLFetch is marginally faster than CloudEvaluate.

logUser=CloudDeploy@APIFunction[{},AppendTo[CloudSymbol@"list1",$RequesterWolframUUID]&];
URLFetch@logUser // AbsoluteTiming

{2.500356,{"0e2497dc-9281-48f3-8e84-14b5e2587446", "0e2497dc-9281-48f3-8e84-14b5e2587446", "0e2497dc-9281-48f3-8e84-14b5e2587446"}

However, this doesn't work when running in the cloud.

CloudEvaluate[URLFetch@logUser // AbsoluteTiming] // AbsoluteTiming

(* errors, because the page was a login prompt *)

I tried CloudEvaluate-ing CloudConnect first, with no luck.

Also, URLFetch and URLFetchAsynchronous are inconsistent.

URLFetchAsynchronous[logUser,#&]

URLFetchAsynchronous[CloudObject["https://www.wolframcloud.com/objects/b2305e35-a643-4a52-a94d-575e0019b0d2"],#1&]

URLFetchAsynchronous[URLBuild@logUser,#&]

AsynchronousTaskObject["https://www.wolframcloud.com/objects/b2305e35-a643-4a52-a94d-575e0019b0d2", 5, 15817848933037716543]

But even putting the syntax aside, handling the "data" event revealed that URLFetchAsynchronous is getting the login prompt, as in the cloud, so the function was not evaluated.

Lastly, the documentation says PutAppend has been updated to work with cloud objects, but I don't know how to use it.

list2 = CloudPut[$RequesterWolframUUID, "list2"];
$RequesterWolframUUID >>> list2 (* fast, no errors *)
CloudGet@"list2"

$RequesterWolframUUID

PutAppend[$RequesterWolframUUID, URLBuild@list2]

OpenAppend::noopen: Cannot open https://www.wolframcloud.com/objects/user-0e2497dc-9281-48f3-8e84-14b5e2587446/list2.

Ideas?

$\endgroup$
6
$\begingroup$

File operations do exactly what I need. Read/write are plenty fast.

Writes:

f := CloudEvaluate[(
  o = OpenAppend["user-call-log"];
  Write[o, $RequesterWolframUUID];
  Close[o]) // AbsoluteTiming];

f & /@ Range@5

{{0.006766, "user-call-log"}, {0.006421, "user-call-log"}, {0.006456, "user-call-log"}, {0.006865, "user-call-log"}, {0.006581, "user-call-log"}}

Reads:

CloudEvaluate[(
  i = OpenRead["user-call-log"];
  ReadList[i]) // AbsoluteTiming]

{0.004605, {"0e2497dc-9281-48f3-8e84-14b5e2587446","0e2497dc-9281-48f3-8e84-14b5e2587446","0e2497dc-9281-48f3-8e84-14b5e2587446","0e2497dc-9281-48f3-8e84-14b5e2587446","0e2497dc-9281-48f3-8e84-14b5e2587446"}}

I hope this is helpful to others.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.