3
$\begingroup$

I have a list of tweetIDs and would like to download them all at once. I tried ServiceConnect["Twitter"], but it limits to one tweet per call and is mighty slow (~3s per call/tweet). Twitter provides APIs for multiple downloads at once but it limits to downloading 100 at one time. I used the Oauth functions in mathematica but it needs verification after every single call as I am using the consumer key instead of access token. I am unable to find the right keywords to specify the access tokens. Can someone help me with this? Here is the call that I am making:

url = "https://api.twitter.com/1.1/statuses/lookup.json?id=" <> StringRiffle[tweetIDs[[;; 100, 1]], ","];
token = OAuthSigning`OAuthToken[
   "ConsumerKey" -> "CONSUMER_KEY",
   "ConsumerSecret" ->"CONSUMER_SECRET",
   "RequestEndpoint" -> "https://api.twitter.com/oauth/request_token",
   "AuthorizeEndpoint" -> "https://api.twitter.com/oauth/authorize",
   "AccessEndpoint" -> "https://api.twitter.com/oauth/access_token"];
URLFetch[url, "OAuthAuthentication" -> token]

This works but needs to be verified for access every single time. I also tried this:

token2 = OAuthSigning`OAuthToken[
       "OAuthVersion" -> "1.0a",
       "ConsumerKey" -> "CONSUMER_KEY",
       "ConsumerSecret" ->"CONSUMER_SECRET",
       "AccessToken" -> "ACCESS_TOKEN",
       "AccessTokenSecret" -> "ACCESS_SECRET"
       "RequestEndpoint" -> "https://api.twitter.com/oauth/request_token",
       "AuthorizeEndpoint" -> "https://api.twitter.com/oauth/authorize",
       "AccessEndpoint" -> "https://api.twitter.com/oauth/access_token"];

But I ended up getting a bunch of errors saying: AccessToken, AccessTokenSecret give values for things that are not valid OAuth 1.0a properties.

In[191]:= $Version

Out[191]= "11.0.1 for Mac OS X x86 (64-bit) (September 21, 2016)"

Any solutions for this?

$\endgroup$
3
$\begingroup$

Presently Twitter paclet does not support https://api.twitter.com/1.1/statuses/lookup.json end point. I have asked the development team to consider adding this request in future versions. Meanwhile one of the developer suggested me a rather simple way of calling this endpoint from mathematica.

In[1]:= twitter = ServiceConnect["Twitter", "New"]

Out[1]= ServiceObject["Twitter", 
 "ID" -> "connection-22ae375a81799332522a0d97ecfc6b2d"]

In[2]:= url = "https://api.twitter.com/1.1/statuses/lookup.json";

In[3]:= ids = (twitter["UserReplies", "Username" -> "donttrythis"])[
    All, "ID"] // Normal;

In[4]:= twitterids = StringRiffle[ids, ","]

Out[4]= "821178765547999232,819195588503867392,818998513979695104,\
818996102795968512,817181239027216386,816816700263796737,\
816766917583118336,815745064185888768,815357650263121920,\
815045109720829952,815043786703147009,815041827854319616,\
815041490384613376,815034501856268289,815031188595220480,\
813474939189673989,813118628262334464,813118334388420608,\
812387175572508672,811394345194557440"

In[5]:= urlstring = URLBuild[{url}, {"id" -> twitterids}];

In[6]:= data = OAuthClient`oauthdata[twitter, urlstring];

In[7]:= res = (<|"ID" -> #["id_str"], "Text" -> #["text"]|>) & /@ 
   ImportString[data, "RawJSON"] // Dataset

Out[7]= Dataset[ <> ]

Out[7] looks like below:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.