43
$\begingroup$

The Import command lets you do a HTTP GET from a URL, like this

Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg"

This sends an HTTP GET request to the server and presumably parses the response according to some rules for inferring the type of data, from the content or from the extension ".jpg" or whatever.

Listening on the socket, we see something like this

GET http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg HTTP/1.1
User-agent: Mathematica/8.0.4.0.0 PM/1.3.1
Host: merkosoncampus.com
Proxy-Connection: Keep-Alive

whereas what I want is something along the lines of

POST http://foo.bar.com/myService HTTP/1.1
User-Agent: Mathematica/8.0.4.0.0 PM/1.3.1
Host: foo.bar.com
Accept: */*
Connection: Keep-Alive
Content-Length: 76
Content-Type: application/x-www-form-urlencoded

{ "myJsonStuff" : {"number" : 1} }

I looked for options in Import, praying for something like Method->"POST", but didn't find anything useful.

I'd be grateful for clues, experience, ideas, sample code. In the mean time, I intend to work around it by writing my desired JSON to a file and using curl to POST it, but it's way sub-optimal.

$\endgroup$
33
$\begingroup$

In a post about automated image uploading Arnoud Buzing describes an undocumented and unsupported POST method. It looks like this:

 xml = Import[url, "XML", "RequestMethod" -> "POST", 
              "RequestParameters" -> {"key" -> key, "image" -> image}];

Note: at the time of this answer I was using V8. Since the arrival of URLFetch in V9 I believe URLFetch is the preferred method.

$\endgroup$
  • $\begingroup$ This is a step in the right direction; I think I discovered that "RequestBody" and "RequestBodyString" are both available options, but I am having quite a struggle with Fiddler and with the Apache Jakarta Java documentation. I'll get back to this next week; doesn't seem trivial. $\endgroup$ – Reb.Cabin Mar 17 '12 at 1:14
  • 3
    $\begingroup$ @Reb.Cabin If you manage to discover more options, please do share them in an answer. I have not been able to make multi part POST requests work with Import. $\endgroup$ – Szabolcs Mar 17 '12 at 9:25
  • $\begingroup$ Szabolcs will do. I have an idea on the fiddler problem ( a day of rest often brings fresh insights ). I may just write a new java class to do it. More later. $\endgroup$ – Reb.Cabin Mar 18 '12 at 2:11
  • $\begingroup$ Just remembered this one, which has the necessary java juju meta.mathematica.stackexchange.com/questions/5/… $\endgroup$ – Reb.Cabin Mar 18 '12 at 2:34
19
$\begingroup$

Use URLFetch in Mathematica 9.0.1.

$\endgroup$
  • 1
    $\begingroup$ Can you elaborate a bit on this and perhaps give a short example? $\endgroup$ – Szabolcs Apr 16 '13 at 15:47
  • 11
    $\begingroup$ Here's an example: URLFetch["http://foo.bar.com/myService", "Method"->"POST", "BodyData"->"{ \"myJsonStuff\" : {\"number\" : 1} }"] $\endgroup$ – Todd Gayley Apr 16 '13 at 17:55
16
$\begingroup$

Using the great example of Szabolcs from here, I implemented the following little function that solves my problem. You must give it a List of Rules, and it will send them to the hard-coded web-service endpoint inside. To test it, I implemented a little NodeJs server that reads the JSON body of a POST request, parses it, unparses it, and sends it back. So this is just a very expensive way of converting rules into JSON in MMA, checking their syntax in Node.Js, and sending them back to MMA to be decoded back into native rules.

bounceRules[requestRules_] :=
 Module[{
   requestJsonString,
   url, client, method,
   entity, responseCode, response,
   responseRules, responseExpression},
  JavaBlock[
   requestJsonString = ExportString[requestRules, "JSON"];
   url = "http://127.0.0.1:5000/bounce";
   client = JavaNew[
     "org.apache.commons.httpclient.HttpClient"];
   method = JavaNew[
     "org.apache.commons.httpclient.methods.PostMethod", url];
   entity = JavaNew[
     "org.apache.commons.httpclient.methods.StringRequestEntity",
     jsonString];
   method@setRequestEntity[entity];
   responseCode = client@executeMethod[method];
   If[responseCode === 200,
    (* then *)
    response = method@getResponseBodyAsString[];
    responseRules = ImportString[response, "JSON"];
    (* else *)
    Message[
     bounceRules::httperr,
     responseCode];
    $Failed
    ]]]

My main hangup, btw, was due to the fact that http://localhost inexplicably does not work. I get timeouts and exceptions and all manner of opaque torture from the java stack. Once I switched to http://127.0.0.1 everything just worked.

$\endgroup$
  • $\begingroup$ Unfortunately I don't know any Java, and it was really difficult for me to get that working ... My sources were this and this, both by ragfield, and also Arnoud's replies $\endgroup$ – Szabolcs Mar 18 '12 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.