39 votes

Web Browsing Automation with Mathematica

Here is a package which does what you want: https://github.com/arnoudbuzing/webunit Clone the repository from github, and place the WebUnit folder under ...
28 votes

Upload from Mathematica to YouTube

I built a service connection for this and blogged about it here: Playing with YouTube from Mathematica Here's an example of how this works. Connect to YouTube First install the paclet: ...
  • 45.8k
24 votes
Accepted

Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails

I'm Riccardo, current developer of URLRead in WL and I have some experience working with encoding in WL. I would like to inform you that this is not a bug. In modern versions of mathematica we have ...
21 votes

Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails

Tracing the evaluation (Mathematica 11.1.1) shows that the string is passed to Developer`ReadRawJSONStream which actually produces the messages: ...
20 votes

Building a GitBook with Mathematica

tl;dr This is really quite easy with a toolkit I've been building; this is an example of a GitBook I built with these tools. Preface This answer is gonna build off some stuff I've been developing ...
  • 45.8k
15 votes
Accepted

Bing daily wallpaper

I would use RawJSON to import the information: ...
  • 125k
14 votes

Does Mathematica support deep links?

As it turns out. Yes. And there's an easy way to check (on Mac). Check out this answer: https://stackoverflow.com/questions/49510/how-do-you-set-your-cocoa-application-as-the-default-web-browser/...
  • 45.8k
14 votes
Accepted

Which external services are compatible with GeoServer?

The keywords to search for are "tile server" and "XYZ URL". I was able to find several services compatible with this format. There is a list, complete with previews here: https://leaflet-extras....
  • 228k
14 votes

Which external services are compatible with GeoServer?

Mapbox works. It requires an API key which can be retrieved for free, for information about pricing read this. Mapbox has many beautiful map styles, and also the ability for you to create your own ...
  • 69k
14 votes

How to implement a free version of WebImageSearch?

So after being wrong about Bing being free, we'll pursue my other suggestion, which was to use the ServiceConnect framework. I demonstrate how to use it in general here. For the purposes of this, ...
  • 45.8k
14 votes
Accepted

Get title of a webpage only?

As far as I am aware, HTTP doesn't support requesting the <title> element specifically. The browser can either GET the ...
  • 8,868
13 votes

How can I commit files to Github by Mathematica?

Another option for individual files and stuff is to use the GitHub API with a custom plug-in to the built-in service connection framework. Here's a service connection I wrote for it: ...
  • 45.8k
13 votes

More complete "MutipartData" POSTs using URLFetch

This is a hack, but it works. ...
  • 1,168
13 votes

How to execute JavaScipt on a webpage and then import the result on OSX?

In this case it would be far more reliable to request the information from the source directly. You can find the source using your web browser's developer tool, in Google Chrome you find this ...
  • 69k
13 votes

Connect to Socket opened by other applications

This is the best use of SocketConnect that I can think for now. ...
  • 31.2k
13 votes

Get title of a webpage only?

The HTTP protocol defines the Range header that can be used to limit the number of bytes returned in a response. Web sites are not required to support this header -- but if the target web site does ...
  • 67.5k
13 votes
Accepted

A tour of AWSLink?

I haven't used these submarined functions for EC2, but I have found the S3 functionalities in this package to be extremely useful. Since there is currently no documentation in v11.3, I thought at the ...
  • 30.8k
13 votes
Accepted

Import data from dynamically generated webpage

Since the page is generated asynchronously, you can use the same data source that the page itself does. Using the network inspector in my browser, I discovered that the page loaded its data from this ...
  • 12.7k
13 votes

How to work with ElasticSearch in Mathematica?

Create the new index test and put the data in it: ...
12 votes
Accepted

How to download pictures from webpage automatically?

Based on this article: kylen314.com/archives/1647 ...
  • 1,923
12 votes
Accepted

How do you submit an Input type of "file" using URLExecute?

As pointed out yesterday by ZachB in answer to another question, there is a hack that makes URLFetch add a filename to the Content-Disposition header as required by RFC 7578 multipart/form-data ...
12 votes
Accepted

How to make a picture grid of all Nobel Laureates in physics?

Here's something to get you started down to path of scraping the somewhat larger individual pictures from the Nobel website: ...
  • 63.1k
12 votes
Accepted

Temporarily simulate being offline programmatically

You can set $AllowInternet to False (or just turn off your WiFi or unplug the cable).
  • 228k
12 votes

Easily download SE posts as notebooks?

Here is an approach, not battle tested, YMMV. It depends on the new body_markdown parameter in the SE API responses. From there we can use whatever MD->NB tool ...
  • 45.8k
11 votes

What is the equivalent of curl?

Assuming you're using Basic Authentication (vs. say, NTML Auth), the following code should work: ...
  • 111
11 votes

Get title of a webpage only?

Import["http://wolfram.com", "Title"] "Wolfram: Computation Meets Knowledge" ...
  • 351k
11 votes
Accepted

Can I trust URLSubmit?

Bug confirmed: [...] The source of the issue unknown to us at this point; however, it seems that all the request are submitted, yet tasks finishes prematurely, before getting the status code. [...] ...
  • 133k
11 votes
Accepted

Mathematica equivalent of a curl snippet

You can use a program like Fiddler to compare the requests being generated by both: Your call to curl generates a request like this: ...
10 votes
Accepted

Parsing JavaScript Script from XML Trees

You have misunderstood how this works. What happens is that JavaScript calls a server-side script which returns the data, and the data is subsequently inserted into the HTML. The data cannot be found ...
  • 69k
10 votes
Accepted

How to parse JSON

The example JSON string: ...
  • 69k

Only top scored, non community-wiki answers of a minimum length are eligible