37

I built a ServiceConnect paclet for this (details on how I did this can be found at the end of this post). You can access it like so: PacletInstall[ "ServiceConnection_NASA", "Site"-> "http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer" ] (note that unless you have BTools installed and have loaded the PacletTools package (e.g. by just ...


29

This might not work for you but is an example to use the Gmail mail server to send emails from a notebook. The example code overrides all settings in the MMa email preference settings and should work out of the box. NB I have tested this only on my Mac. SendMail[ "To" -> "YYY...@me.com", "Subject" -> "Example Message", "Body" -> "My text", "...


28

Using the (deprecated but easy) API documented here, result = Import[ "https://ajax.googleapis.com/ajax/services/search/images?v=1.0&q=fuzzy%20monkey", "JSON"] Import["url" /. ("results" /. ("responseData" /. result))[[1]]] If you hit the service too many times, it'll lock you out for a while. Here's another example that gives the 24th-32nd results:...


25

The following guide shows how to conduct communication between nanohttpd, an http server for Java, and Mathematica. The result is a server that, if you go to its address in a web browser, displays the result of SessionTime[], i.e. the time since the Mathematica kernel associated to the server started. I'm going to write as if the reader was using OS X with ...


23

Here's a set of functions that allows to do this. The code uses many ideas found on this site and on other places on the web. It is a bit factorized already so it should be easily reusable. More on YQL and available tables here: https://developer.yahoo.com/yql https://github.com/yql Query test Edit: this API is great also and simple http://www.quandl.com/...


19

Template Notebook I sunk some time into making a good template notebook for integrating with all this. It lives here: https://github.com/b3m2a1/mathematica-BTools/blob/master/Resources/Templates/ServiceConnectionTemplate.nb Original: If you just want the code I used to generate the paclet it lives here: PacletInstall["CustomServiceConnection", "Site"-&...


18

Something like this: arXiv = ServiceConnect["ArXiv"]; articles = arXiv["Search", {"Query" -> {"Abstract" -> {"Eigenvalues", "Mathematica"}}, MaxItems -> 25, "SortBy" -> "DateSubmitted"}]; articles[All, {"URL", "Title", "Published"}] Looking at the help page has more info: ArXiv


17

Code See this for the main issue that made it take so long to get this up If you just want the service connection you can get it like this: PacletInstall["ServiceConnection_GoogleDrive", "Site"->"http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer" ] Post So this turned out to be a bit of a bear to get the details right for, but not ...


15

This is a complicated task and I believe that Mathematica is not the best tool to do it. If you want to do it only once, just go ahead with the method described in your question. Otherwise, if it is a frequent task, try to select a better tool such as Apache Nutch, which has a built in crawler. Not so easy to get used to, but it saves a lot of time ...


12

So after being wrong about Bing being free, we'll pursue my other suggestion, which was to use the ServiceConnect framework. I demonstrate how to use it in general here. For the purposes of this, though, here's what all you need to do. Get the paclet by running this: (* If you've already installed, replace this with PacletUpdate to get the newest ...


12

This will be an answer in two parts. The first is how to build out a sophisticated, stateful ServiceConnection. The second is how to use this to build out an interactive card game. Just to whet your appetite, here's what the final output looks like: Building a stateful ServiceConnection Prerequisites I'm going to use a toolchain I've developed, but ...


12

Create the new index test and put the data in it: MapIndexed[ URLExecute@HTTPRequest[ <| "Domain" -> "localhost", "Port" -> 9200, "Path" -> {"test", "external", ToString[First@#2]} |>, <| Method -> "PUT", "Body" -> ExportString[<|"Text" -> #1|>, "PythonExpression", CharacterEncoding -> "...


11

After looking better into the comment by Szabolcs, I have chosen this path (which I did try for images, but it didn't cross my mind to use it for web): result = Import[ "https://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=furry%20beasts", "JSON"] and then "url" /. ("results" /. ("responseData" /. result)) to get the links to the pages. Or, ...


11

Update: Full Flow So after much wrangling here is the full OAuth 1.0 flow. (* Get the token *) $consumerKey = "key"; $consumerSecret = "secret"; $oauthTokenResponseString = URLRead[ OAuthSigning`Private`HMACSha1SignatureService[ "http://www.khanacademy.org/api/auth2/request_token", "HMAC", "GET", $consumerKey, $consumerSecret, ""(*token ...


10

Mathematica now supports a native connection to GoogleCustomSearch API. To use image search you can do: gs = ServiceConnect["GoogleCustomSearch"] gs["Search",{"Query"-> "Jennifer Lawrence","SearchType"-> "Image"}] To use GoogleCustomSearch API you need an API Key and a Custom search engine ID. To get the API Key you first have to go to https://...


10

Fun Example I've made use of that here. I stuck the notebook in an accordion with an opener-thumb so I could generally display the page with it closed: But people reading the tutorial can click on the thumb to open a notebook in which to run the code: Explanation Yes! And it's easy! The trick is to use the notebook that gets loaded here: "https://...


10

I did this one by request from Szabolcs. It was a little tough and my implementation is still only in the 1.0.0 stage, but if the demand is there I can polish up the parts of it that remain too rough. To get started, we can install the ServiceConnection like so: PacletInstall[ "ServiceConnection_HouseOfGraphs", "Site" -> "http://www.wolframcloud.com/...


9

Since Mathematica 10 it's possible to pull data from Google Analytics using ServiceConnect and ServiceExecute. I refer you to the documentation for all the details, but here is a small sample. Establish a connection: ga = ServiceConnect["GoogleAnalytics"] Get the number of visitors and the number of sessions over the last ten days: req1 = ServiceExecute[...


9

The following is a sample implementation of a simple HTTP server in Wolfram Language code only: https://github.com/arnoudbuzing/wolfram-server You send it a POST request where the body data of the HTTP request contains the Wolfram Language code you wish to evaluate. The (running) wolframserver.wls script processes the request by evaluating the code string ...


9

I haven't used these submarined functions for EC2, but I have found the S3 functionalities in this package to be extremely useful. Since there is currently no documentation in v11.3, I thought at the very least I could share a brief guide to the most important operations. Setup AWS CLI AWSLink uses the AWS command line tool under the hood, so you need ...


8

Here is something I quickly came up with after a cursory reading of the docs: $alphaVantageKey = (* insert your API key here *); alphaVantageTradingChart[symbol_String, opts___] := Module[{daily, meta, msg, raw, series}, daily = StringTemplate["https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&datatype=json&outputsize=compact&...


7

This so called "cloud cdf" (which should probably be abandoned) is clunky and slower than a turtle, extra points for a way to increase fps. Only a faster internet connection can save you as almost everything has to be calculated cloud side. It may happen that what you want to calculate takes longer than the call itself but generally the connection is the ...


7

I just logged into Google Analytics and had a look at the export formats they offer. They offer many different formats that can be read by Mathematica: CSV, TSV and Excel are those that stand out as very easy to work with. Here is an example where I chart the number of visitors over a certain period of time: data = ImportString["Day Index,Visits 12/11/...


7

Mathematica now supports a native connection to GoogleCustomSearch API. You can do for example gs = ServiceConnect["GoogleCustomSearch"] gs["Search", {"Query" -> "Jennifer Lawrence"}] You can also use image search gs["Search",{"Query"-> "Jennifer Lawrence","SearchType"-> "Image"}] To use GoogleCustomSearch API you need an API Key and a Custom ...


7

Since Mathematica 11.3 there is an experimental implementation for WebDriver, a proposed W3C standard that currently supports Chrome and Firefox web browsers. It allows actions such as "OpenWebPage", "CaptureWebPage" (Image), "JavascriptExecute", and mouse operations such as "ClickElement" and "HoverElement" among others. There is no documented option to ...


7

You can use "ID" to construct a URL to link to the references page on ui.adsabs.harvard.edu: refsLink = Hyperlink["https://ui.adsabs.harvard.edu/#abs/arXiv:" <> First[StringSplit[#, "v"]] <> "/references"] &; ds1 = articles[All, {"ID", "Title"}]; ds2 = articles[All, Key["ID"] /* <|"References" -> refsLink|>]; Join[ds1, ds2, ...


6

After @Hans comment, get much more clear how to do it in webMathematica. It could't be simpler. IMO, the best way to see how it works, after install webMathematica, is going into: tomcatDirectory\webapps\webMathematica\Examples\AJAX. Let's see for example the ReturnDate.jsp file: <msp:evaluate> If[ $$fullDate === "true", DateString[], ...


6

If you really want to go this way then the following is a starting point; however in the comments there are good points why this is not the best solution. r = Compile[{{w, _Real}, {h, _Real}, {time, _Real}, {i, _Integer}}, (0.4 (w + h)) Cos[(time + i) (0.05 + Sin[time*0.00002]/Pi 0.2)]/Pi] w = 1.; h = 1.; Manipulate[Graphics[ Table[Point[...


6

This is a long-ish answer with very little core code. I stuck all of it in a block at the end So to keep this applicable to before v11.1 we'll use Encode over EncryptFile (note that Encrypt would work fine here, if we simply access then re-encrypt a password expression at each usage). First we'll choose a location and name for our encoded file: $...


5

Starting in Mathematica 11.2, there is an experimental built-in function SocketListen that can start a web server and respond to HTTP requests. Further reading: Network Programming Guide.


Only top voted, non community-wiki answers of a minimum length are eligible