13
$\begingroup$

UPDATE 9 March 2017: I added a bounty to this question, I think it is a hard and challenging problem and can prove that Mathematica is capable of almost anything.

Sadly user1473011's answer does not work. I came back to this issue for many times, so far all my attempts to handle this from within Mathematica failed. Although I have a working solution in the form of a Java program I would rather have it done inside Mathematica.

With Mathematica 11.0.1 on Windows 10 I want to use URLFetch to do a so called 'Non-interactive/Bot login' at Betfair. The Betfair API manual gives this example in Python:

#!/usr/bin/env python

import requests

payload = 'username=myusername&password=password'
headers = {'X-Application': 'SomeKey', 'Content-Type': 'application/x-www-form-urlencoded'}

resp = requests.post('https://identitysso.betfair.com/api/certlogin', data=payload, cert=('client-2048.crt', 'client-2048.key'), headers=headers)

if resp.status_code == 200:
  resp_json = resp.json()
  print resp_json['loginStatus']
  print resp_json['sessionToken']
else:
  print "Request failed."

'client-2048.crt' and 'client-2048.key' are files which I have generated with win-openssl and are stored on disk. I have a username and password.

There are also examples for C# and Java in the API Manual

I came as far the following URLFetch but I get a HTTP 405 return code, which was to be expected because I did not know how to handle the certification file, I generated earlier, basically the question is how to translate the python to URLfetch with an emphasis on the cert file.

URLFetch["https://identitysso.betfair.com/api/certlogin", {"StatusCode",    "Headers", "Content"}, "Headers" -> {"X-Application" -> "nilostep", "Accept" ->"application/json"}, "Parameters" -> {"Username" -> "myusername", "Password" -> "mypassword"}];

I have successfully used the API before but I copied a 'session token' for access which you can get only after manually logging in. Related question: How to access a web API with HTTP POST and JSON

(EDIT: Any example of a https secure login using certification keys using Mathematica will help.)

(EDIT-2: Might this help?: Can anyone explain how to use Mathematica's OAuth package? ? )

$\endgroup$
  • $\begingroup$ No, typo. I'll correct it. Thanks @Kuba $\endgroup$ – nilo de roock Feb 12 '16 at 11:13
  • $\begingroup$ You should remove the space before "https" in URLFetch[" http. $\endgroup$ – rhermans Feb 12 '16 at 12:03
  • $\begingroup$ I am embarrassed but I can move on now. Thanks @rhermans $\endgroup$ – nilo de roock Feb 12 '16 at 12:08
  • $\begingroup$ Basically the question is how to translate the python to urlfetch. $\endgroup$ – nilo de roock Feb 12 '16 at 12:16
  • $\begingroup$ Presumably "session Token" is a cookie ...right? So have you tried using the "Cookies" and "StoreCookies" option? $\endgroup$ – Mike Honeychurch Feb 13 '16 at 5:16
3
$\begingroup$

This api isn't accessible in the US, which makes it difficult to post a working code.

Try:

myCert = Import["path/to/my.crt"];

file = FileNameJoin[{
                     Replace["Location",PacletInformation["HTTPClient"]],
                     "SSL",
                     "cacert.pem"
                     }];

(*Append your certificate to the existing cacert.pem file*)
o = OpenAppend[file];
WriteString[o, "\n" <> myCert];
Close[o];

URLFetch[ "https://identitysso.betfair.com/api/certlogin",
    { "StatusCode", "Headers", "Content" },
    Method->"POST",
    "Headers" -> {
        "X-Application" -> "nilostep",
        "Content-Type" -> "application/x-www-form-urlencoded",
        "Accept" -> "application/json", 
        "Username" -> "myusername",
        "Password" -> "mypassword"
    }
]
$\endgroup$
  • $\begingroup$ There are other access points : identitysso.betfair.com identitysso.betfair.es identitysso.betfair.it identitysso.w-con.betfair.com identitysso.betfaironline.eu $\endgroup$ – nilo de roock Feb 15 '16 at 18:21
  • $\begingroup$ I have created files with the following exts: crt, csr, key, p12 and pem. In the workaround that I created ( see answer ) I use the file with p12 extension but this file requires an additional password. My point being which file should I use in "path/to/cert" ? I used the path to .pem and received the following result: {400, {{"Content-Type", "text/html;charset=utf-8"}, {"Content-Length", "82"}, {"Date", "Mon, 15 Feb 2016 19:49:52 GMT"}, {"Connection", "close"}, {"Vary", "Accept-Encoding"}}, \ "<html><head><title></title></head><body><h1>HTTP Error Code \ 400</h1></body></html>"} $\endgroup$ – nilo de roock Feb 15 '16 at 19:52
  • $\begingroup$ Why haven't you added 'Content-Type': 'application/x-www-form-urlencoded' ? See Python example. $\endgroup$ – nilo de roock Feb 15 '16 at 20:29
2
$\begingroup$

The following solution is a working workaround.

Needs["JLink`"]
BOTA = "D:\\Dropbox\\DATA\\MyJavaProjects\\bota\\out\\artifacts\\bota_jar\\bota.jar";
ReinstallJava[CommandLine -> "C:\\Progra~1\\Java\\jdk1.8.0_45\\jre\\bin\\java", ClassPath -> BOTA];
conn = JavaNew["org.nilostep.bota.HttpClientSSO"]
conn@getSessionKey[]

bota.jar contains the class HttpClientSSO as described in the API manual ( see question ) with an additional method that returns the sessionkey as a string.

Classpath is the path to bota.jar

getSessionKey() returns a String and is a new method in HttpClientSSO

The above should be possible with just URLFetch though.

$\endgroup$
2
$\begingroup$

On SSL and cert files

So this seems to fundamentally be a question of doing SSL certification. Going from here there's some discussion of adding SSL certificates in a standard location. Here's a way to find your equivalent of that file:

$CAFile =
  First@FileNames["*.pem", $InstallationDirectory, \[Infinity]];

Mine lives here:

FileNameJoin@{$InstallationDirectory, "SystemFiles", "Links", 
  "CURLLink", "SSL", "cacert.pem"}

And as URLRead calls URLFetch, which in turn calls the CURLLink system I think this is the right place to append my credentials.

Here's a wholly untested way to do this (as I have no .pem files to test it with) without copying it in by hand:

CAAppend[cert_String?(Not@*FileExistsQ)] :=

  If[Not@StringContainsQ[Import[$CAFile, "Text"], cert],
   CopyFile[$CAFile,
    FileNameJoin@{DirectoryName@$CAFile,
      FileBaseName@$CAFile <> "_backup." <> FileExtension@$CAFile
      }];
   With[{append = OpenAppend[$CAFile]},
    WriteString[append, cert];
    Close@append;
    ]
   ];
CAAppend[cert_String?FileExistsQ] :=
  CAAppend@Import[cert, "Text"];

Obviously the best way would be to find some way to pass the necessary SSL cert data in the headers of an http request, but that'll take more research.

Original posting

Since you mention you're on 11. have you tried formatting an HTTPRequest instead? I've had success using it to get an application-only access token off of Twitter. Basically I followed their guide and ended up with this:

HTTPRequest[
  <|
   "Scheme" -> "https",
   "Domain" -> "api.twitter.com",
   "Path" -> {"oauth2", "token"}
   |>,
  <|
   "Host" -> "api.twitter.com",
   "Body" ->
    {"grant_type" -> "client_credentials"},
   "UserAgent" ->
    $consumerName,
   "Headers" -> {
     "Authorization" ->
      "Basic " <> $bearerToken
     }
   |>
  ] // Import

then translating that over to the template you had I ran this (note that when you have your real $cert files you should uncomment File/@ in $cert.

$cert =
  (*File/@*){"client-2048.crt", "client-2048.key"};
$username =
  "myusername";
$password =
  "mypassword";
$application =
  "SomeKey";
HTTPRequest[
  <|
   "Scheme" -> "https",
   "Domain" -> "identitysso.w-con.betfair.com",
   "Path" -> {"api", "certlogin"}
   |>,
  <|
   "Headers" -> {
     "X-Application" ->
      $application
     },
   "Body" -> {
     "username" ->
      $username,
     "password" ->
      $password,
     "cert" ->
      $cert
     }
   |>
  ] // Import[#, "JSON"] &

Executing that returns:

{"loginStatus" -> "ACCOUNT_ALREADY_LOCKED"}

Which tells me that something is happening. Try dropping in your credentials and see what happens.

$\endgroup$
  • $\begingroup$ I received, the {"loginStatus" -> "CERT_AUTH_REQUIRED"} message. My $cert = {"C:\\OpenSSL-Win32\\client-2048.crt", "C:\\OpenSSL-Win32\\client-2048.key"} these files exist in my dir. I have also tried it with $cert = {"C:\\OpenSSL-Win32\\client-2048.p12"}, this is the file I use in my Java program. No success either. I also tried $cert = "C:\\OpenSSL-Win32\\client-2048.p12" with no success. $\endgroup$ – nilo de roock Mar 11 '17 at 19:54
  • $\begingroup$ I would try mapping File over those as is shown in the HTTPRequest docs. $\endgroup$ – b3m2a1 Mar 11 '17 at 19:55
  • $\begingroup$ I thought that was a comment. Will try now. $\endgroup$ – nilo de roock Mar 11 '17 at 19:56
  • $\begingroup$ I had it commented out because HTTPRequest whined at me, thinking I was trying to send files that didn't exist. If you look under the "Body" part of Details and Options it has something on using File to send files. $\endgroup$ – b3m2a1 Mar 11 '17 at 19:57
  • $\begingroup$ OK. Message[Import::fmterr, "JSON"]; Return[$Failed] Import::fmterr: Cannot import data as JSON format. If I leave out the "JSON" from the Import then I get HTTP Error Code 400 $\endgroup$ – nilo de roock Mar 11 '17 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.