Accessing the Twitter API using Mathematica 11

Question

The built-in social media functions are limited, so I'm using Twitter's web API for my work.

The following function worked for me when I was using versions of Mathematica earlier than 10.3.

token=HTTPClient`OAuthToken[HTTPClient`OAuth`Private`OAuth10Parameters[ 
    "api.twitter.com", None, Automatic, "verifier", Automatic, Automatic, 
    consumerKey, consumerSecret,
    "https://api.twitter.com/oauth/request_token", 
    "GET", "https://www.twitter.com/oauth/authorize",
    "RemoveWhitespace", "https://api.twitter.com/oauth/access_token", 
    "GET", "HMAC-Sha1", "HMAC", "oob", "Text/1.0", "Text/1.0", None],
    HTTPClient`OAuth`Private`Token10[ accessToken, accessTokenSecret]];
url = "https://api.twitter.com/1.1/search/tweets.json";
URLFetch[url, "OAuthAuthentication" -> token]

But when I upgraded to Mathematica 11, that code stopped working and gave error messages like the following:

OAuthSigning`OAuthSignURL::nopt: The option OAuthAuthentication is missing in OAuthSigning`Private`OAuthURLFetchList[https://api.twitter.com/1.1/search/tweets.json?count=100&q=%23brexit, {CredentialsProvider -> None, OAuthAuthentication -> OAuthSigning`OAuthAuthentication[HTTPClient`OAuthToken[HTTPClient`OAuth`Private`OAuth10Parameters[api.twitter.com, None, Automatic, <<15>>, Text/1.0, None], <<1>>]]}].

I also tried the following:

token = HTTPClient`OAuthAuthentication[
    "ConsumerKey" -> "your_api_key", 
    "ConsumerSecret" -> "your_api_secret", 
    "RequestEndpoint" -> "https://api.twitter.com/oauth/request_token",
    "AuthorizeEndpoint" -> "https://api.twitter.com/oauth/authorize", 
    "AccessEndpoint" -> "https://api.twitter.com/oauth/access_token"];
url = "https://api.twitter.com/1.1/search/tweets.json";
URLFetch[url, "OAuthAuthentication" -> token]

I need my program to be up to date with the latest versions of Mathematica. Any idea where might the problem be?

Answer 1

Update

Here's a cleaner rewrite of my original:

First authenticate:

$so = ServiceConnect["Twitter", "New"];

Then extract:

ServiceConnections`Private`serviceAuthentication[$so[[-1, -1]]][[2]]

OAuthSigning`Private`Token10[
 "806596495344603136-CEWIkGjWYevKFKsUT7uJ5XCLoLWyON2",
 "NuE2kf4PjPNkzUCpb6dASqjMfMoXbE07ACTj7YImeMb1R"
 ]

Note, too, that if you just want to add requests to the Twitter service connection you can do that.

First do this:

FileNameJoin@{
   PacletFind["ServiceConnection_Twitter"][[1]]["Location"],
   "Kernel",
   "Twitter.m"
   } // SystemOpen

And add your requests as described here

Note that these won't persist if WRI updates its connection though, in which case you'll have to copy the requests to the new paclet.

---

Update 2:

By the way, I think your original request would now look like:

token =
  OAuthSigning`Private`OAuthFlow[
   OAuthSigning`Private`OAuth10Parameters["api.twitter.com", None, 
    Automatic, "verifier", Automatic, Automatic, "consumerKey", "consumerSecret", 
    "https://api.twitter.com/oauth/request_token", "GET", 
    "https://www.twitter.com/oauth/authorize", "RemoveWhitespace", 
    "https://api.twitter.com/oauth/access_token", "GET", "HMAC-Sha1", 
    "HMAC", "oob", "Text/1.0", "Text/1.0", None
    ],
   OAuthSigning`Private`Token10["accessToken", "accessTokenSecret"]
   ];

---

Original

Rather than go down to the metal and do the auth oneself, you could also rely on Mathematica's built-in ServiceConnect and then scrape the necessary auth data from that.

Some spelunking led me to ServiceConnections`Private`serviceAuthentication which takes a UUID and returns the associated auth info, if the service is authenticated. So this a feasible way to get that data.

It returns an OAuthSigning`OAuthToken object which appears to be the OAuth parameters and the token.

Using the following function I was able to get auth data that may be usable:

With[{service = ServiceConnect["Twitter"]},
 While[
  ("ID" /. Last@service // 
     ServiceConnections`Private`serviceAuthentication) === {},
  "Lock"];
 With[{data = ("ID" /. Last@service // 
      ServiceConnections`Private`serviceAuthentication)},
  <|
   "Parameters" -> First@data,
   "Token" -> Last@data
   |>
  ]
 ]

this returned and object that looked like OAuthSigning`OAuthToken[<<"Twitter OAuth 1.0">>] which is an elided form for the following:

OAuthSigning`OAuthToken[OAuthSigning`Private`OAuth10Parameters["Twitter", None, 
  Automatic, "verifier", Automatic, OAuthClient`oauthChannelVerify[#1, 
    {"Twitter", "connection-6f1c5e584c496a5101d61d0925eeadea"}] & , "Wolfram", "Token", 
  "https://api.twitter.com/oauth/request_token", "GET", 
  "https://api.twitter.com/oauth/authorize", "RemoveWhitespace", 
  "https://api.twitter.com/oauth/access_token", "GET", "HMAC-Sha1", "HMAC", "https://ww\
w.wolfram.com/oauthlanding/connector/?connectionid=connection-6f1c5e584c496a5101d61d092\
5eeadea", "Text/1.0", "Text/1.0", None], OAuthSigning`Private`Token10[
  "806596495344603136-CEWIkGjWYevKFKsUT7uJ5XCLoLWyON2", 
  "NuE2kf4PjPNkzUCpb6dASqjMfMoXbE07ACTj7YImeMb1R"]]

I am not certain this will suffice for your purposes and this obviously requires a login step that you may not want.

The best option, however, may be to just write your own Twitter connection. I was able to build a connection to the StackExchange AP with OAuth and all in just a few hours and if you really need the flexibility, this is obvious the most flexible solution.

As dsingh points out, any solution that doesn't use toplevel/widespread functionality is likely to be unstable and so if you want a dependable solution that will work across versions/subversions of Mathematica, you're more-or-less restricted to building it yourself.