Long time reader, first time writer.
I've been following the presidential candidates' popularity on Facebook as a side project recently, and storing the total number of likes on their page daily on Data Drop (and actually presented on the work at WTC2015). I'm trying to migrate a Java script to a daily ScheduledTask
via CloudDeploy
that uses DatabinAdd
to update the like counts in a Databin
.
Here's the code I'm trying to use (I omitted some candidates because that part just adds length to the text). I've tried several variants and nothing seems to get a 100% solution.
With[{shortid = "XXXXXXX",
candidates = <|"Joe Biden (D)" -> "http://www.facebook.com/joebiden",
"Lincoln Chafee (D)" -> "http://www.facebook.com/chafee2016", ...|>},
CloudDeploy[ScheduledTask[
Print["Hey, I got here..."];
getLikeCountForURL[url_String] := Module[{},
Import["http://graph.facebook.com/fql?q=select%20%20like_count%20from%20link_stat%20where%20url=%22"
<> url <> "%22", "JSON"][[1, 2, 1, 1, 2]]];
getLikeCountForAllCandidates[] :=
Module[{}, getLikeCountForURL /@ candidates];
Print["I defined my functions...."];
val = getLikeCountForAllCandidates[];
Print[val];
bin = Databin[shortid];
Print["bin created"];
DatabinAdd[bin, val];
Print["Added to bin"];
, DateObject[{_, _, _, 12, 00}]], "FB Election 1"]
]
The Problem is that the DatabinAdd
command is just not working properly. Originally, I had placed the bin=Databin["..."]
command in the first part of the With
, and would get errors that complained about the $IterationLimit
being hit. So I tried the code above, and I think I just get a timeout of some sort. The log on the Programming Cloud just shows (some repetitive output omitted):
Fri 30 Oct 2015 13:06:52 GMT-05:00 Evaluating expression: Print["Hey, I got here..."]; getLikeCountForURL[url_String] := Module[{}, ... Print["Added to bin"];
Fri 30 Oct 2015 13:06:52 GMT-05:00 === Evaluation started ===
Hey, I got here...
I defined my functions....
<|Joe Biden (D) -> 855099, Lincoln Chafee (D) -> 12040, ... John Kasich (R) -> 149161|>
bin created
And doesn't get to the final point in the code (nor does my Databin
show any new data). I just get an email from the Data Science Platform that says that it was $Aborted
.
I temporarily changed my solution to use the REST-type API instead by using URLFetch
on an appropriate URL (this is what I did in Java before setting the "write" permissions to "Private"). That nearly worked. It required me to provide a username and password, so I did that with URLFetch["...","Username"->"myWolframUsername","Password"->"myPassword"]
and that still didn't work. Apparently it didn't pass the login info correctly? I tried this locally and still got a popup window about needing to provide username and password. Running this in the cloud, I got an error message about InputString
not yet being supported in the cloud (which I think is equivalent to saying "hey, you can't do a popup through the cloud").
So I'm feeling pretty stuck. Anybody have any suggestions for me to try? I know this is a really long question, but I wanted to clarify from the get-go all the things I've tried already. I've spent a couple hours on this already and thought I'd ask you all instead.
Update: So this just ate all my cloud credits. I think like 12,000 or more credits trying to run this like 10 times (failing each time). So yeah. Guess I don't need much of a solution until my monthly refresh in a couple weeks.
CloudDeploy
but in order to continue doing your analysis is there any reason why you cannot simply run this from your desktop with a scheduled task and store the data either in a database, or export as a file or even email yourself the results each day? In other words i their a particular reason why you want to use the cloud? $\endgroup$With[{bin = "XXXXXXXX"}, CloudDeploy[Delayed[DatabinAdd[bin, RandomInteger[{1000, 9999}]]; Values[Databin[bin, "Day"]]], FileNameJoin[{$CloudRootDirectory, "EvanOttDatabinAdd"}]]]
Please confirm that this also works for you. $\endgroup$