# Using URLFetch/URLExecute to post to an internal server

I've been looking up how to do this for a while and I haven't seen any thorough explanations, just hints. Is there like a secret repository of clearly defined Mathematica code somewhere that I cant find? Anyways, I have an element in JSon and want to add it to a internal url specific to that element and for which I have write permissions. For example: https://www.internalSite.com/master/siteID/elementID

What I've been seeing consistently is something like

URLFetch["url", "Method"->"Post", "RawJSON"]


Which is strange because the reference guide says URLFetch only takes in two arguments: the url and "elements." It does mention methods (but not how to use them), but only the "Get" method.

So my question is basically, URLFetch was replaced w/ URLExecute, so how can I use it to "Put" Json on the website? What arguments do I need to specify, how does defining a method work, etc?

i.e. When I go to the above url, I want to see the JSon I put there.

EDIT: I just remembered I'll actually have to create the url as well as add the element to it. We have a server "internalSite.com/master/siteID" for example that contains links to all the JSon elements. I need to create another linked url "https://www.internalSite.com/master/siteID/elementID" for example and add the raw JSon to said url.

Thank you

• I took a look at that. So they go after the main arguments but still within the function URLExecute? I couldnt find anything on the method "put" or "post." So what I am understanding is that if you dont pass a method to URLExecute, then it defaults to "Get?" If you specify "Put" or "Post," then it does those. What else do I have to include? Like where would the JSon go in the code? After the Method? – Joe Blow Mar 21 '17 at 10:46
• check URLRead -> Details&Options section in documentation – Kuba Mar 21 '17 at 10:48
• I have experience with this. I do it all the time with CouchDB. I can prepare a response later today. – Gustavo Delfino Mar 21 '17 at 12:29
• Did you see the comment here regarding the usage of URLFetch for post queries? That's how you do it. Some things could go wrong, for example you say that you have "write privileges" but how is the server going to know that? – C. E. Mar 21 '17 at 12:36

Lets begin by installing an "internal server". CouchDB is a database that uses the HTTP protocol (no DatabaseLink needed here). It works using REST like many other web services out there. If you are on a Mac or PC it is just a matter of running the installer with default options. After this we can confirm that the server is working:

In[1]:= Import["http://127.0.0.1:5984"]
Out[1]= {"couchdb":"Welcome","version":"2.0.0","vendor":{"name":"The Apache Software Foundation"}}


This server always replies in JSON format so we can fetch and parse in one step:

In[2]:= Import["http://127.0.0.1:5984","RawJSON"]
Out[2]= <|couchdb->Welcome,version->2.0.0,vendor-><|name->The Apache Software Foundation|>|>


I use Import[] for fetching because it is the easiest way to generate and parse simple GET requests. But you can also use other commands like URLRead or URLFetch.

Let's create a database. In CouchDB tutorial, they do it like this:

curl -X PUT http://127.0.0.1:5984/baseball


from Mathematica, one way of doing this is:

In[3]:= URLExecute[HTTPRequest["http://127.0.0.1:5984/mydatabase", <|"Method" -> "PUT"|>]]
Out[3]= {"ok" -> True}


By default, CouchDB allows all operations (admin party) but only from the same computer where it is installed. If you add a user/password then you need to provide your credentials or a cookie.

Now lets get a list of the databases in order to verify that "mydatabase" is there:

In[4]:= Import["http://127.0.0.1:5984/_all_dbs","RawJSON"]
Out[4]= {_global_changes, _metadata, _replicator, _users, mydatabase}


It is there. Now let's put in there do some documents. We can do this with PUT requests or with POST requests. With PUT, we have to specify the document ID in the URL:

In[5]:= URLExecute[HTTPRequest[
"http://127.0.0.1:5984/mydatabase/1",
<|"Method"->"PUT",
"ContentType" -> "application/json",
"Body"-> ExportString[<|"random"->RandomInteger[10,10]|>,"RawJSON"]|>]]

Out[5]= {"rev" -> "1-683f50d046d9d04090215fac932eb395", "id" -> "1", "ok" -> True}


In[6]:= Import["http://127.0.0.1:5984/mydatabase/1","RawJSON"]["random"]
Out[6]= {0, 7, 7, 10, 2, 10, 8, 8, 9, 8}


Lets now create a document with a POST request:

In[7]:= newWithPOST =  URLExecute[ HTTPRequest[
"http://127.0.0.1:5984/mydatabase/",
<|"Method" -> "POST",
"ContentType" -> "application/json",
"Body" -> ExportString[<|"random" -> RandomInteger[10, 10]|>,"RawJSON"]|>],"RawJSON"]

Out[7]= {"rev" -> "1-7c2799fa64b5dd6d9d66be7af3f8de15",  "id" -> "0467a21cd602ed6827dc8fad3e0014a2", "ok" -> True}


And now we read it back:

In[8]:= Import[URLBuild[{"http://127.0.0.1:5984/mydatabase/",newWithPOST["id"]}],"RawJSON"]["random"]
Out[8]= {3,8,4,5,0,8,5,10,5,1}


We can also use other commands like URLFetch but it doesn't seem natural to "insert" using a "fetch".

• This was perfect, thank you. I found out today that what was confusing me was that (among other things you covered nicely) our api uses PUT requests to handle multi-element GET requests. If a PUT request has a body, then the api returns the elements listed in the body. I guess they do that because we have no other use for PUT and it's less call intensive than doing each element one by one with GET. – Joe Blow Mar 22 '17 at 4:21
• This is outputting the following error: 411 Length Required nginx – Joe Blow Mar 22 '17 at 18:44

I didn't quite catch the last part about building the URL, but maybe you can use URLBuild[]. About sending requests, however, I think I know the answer:

I had to deal with something similar recently, where I wanted to simulate the requests made by an external API.

To test your requests, you can use Hookbin.com. I've created an endpoint which I can access using:

https://hookb.in/v953z87G

In Mathematica, I've learned to use HTTPRequest[] like this:

url="https://hookb.in/v953z87G";
body=ExportString[Import["/path/to/file.json","RawJSON"],"RawJSON"];

req = HTTPRequest[
url,
<|"Method" -> "POST","Body" -> body,
"ContentType" -> "application/json"|>];

URLExecute[req]


which if done correctly, will return:

{"success" -> True}


by changing the last line, you can control the structure of the results:

URLExecute[req, {}, "RawJSON"]


which will return:

<|"success" -> True|>


Now, refreshing the hookbin page, shows you the result. You should see the JSON content in "Body" section.