The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [web-access]

Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.

Filter by
Sorted by
Tagged with
44
votes
1answer
3k views

Web Browsing Automation with Mathematica

I want to build a bot to automate web browsing, this mean something like: filling forms press "submit" buttons find certain text inside pages and so on... How can I do this with Mathematica? The <...
25
votes
3answers
4k views

Who is to blame: parsing UTF8 encoded JSON HTTPResponse fails

Bug introduced in 11.0 or earlier and persists through 11.3 (confirmed by WRI Support but read summary below) Answers summary I think we need a quick summary here to justify keeping bug. Original ...
48
votes
8answers
2k views

Is it possible to invoke the OEIS from Mathematica?

I had always wondered if there might be a way to write a function, which I'll call OEISData[], that more or less works as a curated data function for The On-Line ...
29
votes
3answers
4k views

Export to Google Spreadsheet with OAuthAuthentication

Context I'm trying to setup a connection between Mathematica and Google Docs. Specifically, I would like to be able to export data to a Google Spreadsheet (Note: if the spreadsheet is public it's ...
7
votes
1answer
354 views

curl in Mathematica, google link-shortener

I want to automatically get short links equivalents for things I CloudDeploy. I'd appreciate some tips on integrating command line workflow below into Mathematica ...
1
vote
1answer
451 views

Is there a better way to download images in parallel?

Is this the best way to batch save a list of image urls in parallel? ...
5
votes
1answer
204 views

Decoding GZIP encoded Body, BodyBytes (ByteArray) and BodyBytesArray from URLRead

When calling the Stack Exchange API v2.2 using the new (Mma11) function URLRead we get a GZIP encoded Body in alternative formats: ...
30
votes
2answers
643 views

Reading from a socket stream

I noticed an undocumented socket package (SocketLink) in Mathematica (Using version 10, not sure when it was actually added) and wrote a barebones HTTP responder ...
19
votes
2answers
1k views

Encrypt the password used for SendMail

I want Mathematica to send me an email when the computation is finished. To that end, I have a cell with SendMail[...] which I place behind my computation in the ...
8
votes
1answer
787 views

More complete “MutipartData” POSTs using URLFetch

I'm trying to upload a file to a wiki using Mathematica's URLFetch. Some preliminary work is required - one needs to log in to the wiki, obtain an edit token, and ...
15
votes
2answers
708 views

How do you submit an Input type of “file” using URLExecute?

I have to submit a form like as follows: <input type="file" name="jpgFile"> I can't seem to find any instructions on sending this in URLExecute. If I ...
8
votes
1answer
193 views

Authentication dialog for URLDownload / URLSave

Bug introduced in 11.0 and persists through 11.3 The original problem is was fixed in V11.2 but since then it returns File[path/to/file/that/does/not/exits] ...
7
votes
1answer
377 views

Programmatic browser interaction: How to click on web-page and get new URL and list of links

In Mathematica it's easy to Import data from the web and perform all sort of Web Operations. But the later doesn't allow to simulate a more interactive browser ...
3
votes
0answers
251 views

URLFetch parameters interfere with POST

(orig: URLFetch Multipart POST fails to post Body?) Update 2: So after much weeping and gnashing of teeth Mathematica deemed me worthy of figuring out (some of) what was going on. It turns out the ...
2
votes
1answer
188 views

How to omit the authentification dialog during URLRead?

When I use URLRead Mathematica will popup a window like: Actually I have given the parameters Password and ...
43
votes
3answers
4k views

How to do http POST in Mathematica?

The Import command lets you do a HTTP GET from a URL, like this Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg" This sends an ...
50
votes
1answer
3k views

Google Map in Notebooks?

GoogleMaps are implemented as HTML and Javascript code which can be viewed by a web browser. Is it possible to load this map into the Mathematica notebook, which obviously means for example that <...
13
votes
3answers
457 views

Which external services are compatible with GeoServer?

Using the GeoServer option, it is possible to load map tiles from external services. Which services are compatible with ...
10
votes
1answer
600 views

How to implement a free version of WebImageSearch?

I really don't like functions that I have to pay for, if they can be implemented for free. Is there a way to get around this pay-wall for WebImageSearch? Perhaps using some other free api or service? ...
10
votes
2answers
131 views

How to show Plots or Graphics in mini MMA HTTP Server?

I learned some codes to make a mini HTTP server in MMA way here. ...
3
votes
3answers
484 views

URLFetch functionality for a Persian website

I want to fetch the content of a Persian website. I tried URLFetch["http://www.tsetmc.com/"] Why is Mathematica unable to recode the characters in the results ...
9
votes
1answer
261 views

Import data from dynamically generated webpage

I'm trying to get data from a web-page data=Import["https://stats.nba.com/team/1610612738/traditional/", "Data"] It returns only a textual information. ...
6
votes
1answer
109 views

Export Data from several tabs in a website

Here is the website from the official federation: http://results.ittf.link/index.php?option=com_fabrik&view=list&listid=36&Itemid=158 I want to export the Data (the table). There are ...
3
votes
1answer
594 views

How can I determine `Parameters` for use in `URLFetch`?

I'm trying to download data from website automatically using Mathematica, and URLFetch seems like it should be suited to the task. I'm interested in its "passing ...
4
votes
2answers
273 views

How to call Picture Bed API

I want to call this Picture bed API to upload my picture and return its link.I have translated the official instructional the page into English.And there is a example succeed to use it with Python. I ...
9
votes
2answers
1k views

How can I commit files to Github by Mathematica?

How can I publish Notebooks to Github via using Mathematica? I'm on windows 8.1 x64, Mathematica 9. How can I upload the current notebook to Github? My function expected is: ...
11
votes
1answer
253 views

How to work with ElasticSearch in Mathematica?

Steps to install and run ElasticSearch: install Java (64-bit): https://www.java.com/en/download/manual.jsp download ElasticSearch (https://www.elastic.co/start) and extract archive in ...
10
votes
0answers
107 views

URL functions overload

All these functions seem capable of doing the same things: Import, URLFetch, URLSave, ...
14
votes
2answers
533 views

Does Mathematica support deep links?

So GitHub sent me on quite the journey, after it opened its desktop app from my browser. Copying the link I found something looking like: "github-mac://...". I ...
8
votes
1answer
183 views

How can I use multiple overlaid map layers (e.g. basemap, labels) in GeoGraphics?

We can use GeoServer to use map tiles from an external service. There are many compatible services: Which external services are compatible with GeoServer? These ...
7
votes
2answers
353 views

Web Browsing Automation: how to expand all comments under a blog post?

I wish to expand all the comments under a LiveJournal post for the purposes of saving the complete thread of comments. According to LiveJournal's FAQ when a post gets more than 50 comments they are ...
16
votes
3answers
373 views

How do I put my packages into Dropbox

I don't want to necessarily put my packages in Mathematica's $UserBaseDirectory, which is on my local hard drive. Can I somehow tell Mathematica to look for ...
10
votes
3answers
666 views

How to make a picture grid of all Nobel Laureates in physics?

I want to make a picture grid of all Nobel Laureates in physics using mathematica. Later I wanna print this picture grid on a big wall! The first thought I came up with is to exploit the powerful ...
7
votes
0answers
575 views

How to crawl some website needed username and password

In version 11,if we want to crawl public website like wiki,we can do it like this: URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"] But I don't know how ...
6
votes
1answer
201 views

Import citation data for BibTeX using DOI?

I recently discovered that Digital Object Identifiers can be used for a lot more than just a reliable link to an article or dataset, and in some cases the DOI url's support semantic queries. This ...
6
votes
1answer
87 views

Inconsistent HTTPResponse handling

It seems that sources of URL* related features are not initialized properly which leads to problems in many cases. Please follow those steps to reproduce the ...
6
votes
1answer
209 views

How to use URLDownload if the url doesn't have the correct filetype extension?

I'm using the new (in V11) function UrlDownload[] I have a url I'd like to save to a local file, but the url is missing a (or has the incorrect) filetype extension....
5
votes
1answer
752 views

How to access a web API with HTTP POST and JSON

I want to get data using HTTPS from POST https://developers.betfair.com/api.betfair.com/exchange/betting/json-rpc/v1 HTTP/1.1 using JSON ...
5
votes
1answer
147 views

How to click a button to upload local file with WebExecute?

A webpage has a "browse" button to open the filesystem to select a file and upload it, in html it looks like this: ...
5
votes
0answers
212 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
3
votes
1answer
107 views

Dynamic WebImageSearch

I am a little bit lost here, I'd be really grateful if someone could help me. :) We are working on a "video" (actually images with transitions). We are having a list of specific words eg.: ...
9
votes
1answer
1k views

How to execute JavaScipt on a webpage and then import the result on OSX?

I would like to import data from this page through Mathematica, http://www.oddsportal.com/soccer/netherlands/eredivisie/ . I tried importing through the default Import function ...
8
votes
1answer
161 views

Inconsistent Authorization Dialog (401) handling

Bug introduced in V11 and fixed in 11.3 [CASE:3855554] was created [...] It does appear that the URLRead is not behaving correctly. I have forwarded an incident report to our developers with ...
8
votes
1answer
416 views

Importing images from the web using ImportString & URLFetch

I can bring images from the web using the Import command like this: Import["http://blog.wolfram.com/wp-content/uploads/2008/06/se-30.jpg"] Sometimes the web ...
6
votes
1answer
200 views

How to use the new function FindCookies?

This is a further question of this post. Actually this code from SqRoots.And this code has completed the task. (I assume you are using Mozilla Firefox and MMA version 11.0) ...
5
votes
1answer
144 views

Cloud CDF and HTTPRedirect

Cross posted on community.wolfram.com Question and requirements Is it possible to HTTPRedirect from a Cloud CDF via e.g. Button...
4
votes
2answers
213 views

Use URLFetch in init.m

I'm not able to use URLFetch in my $UserBaseDirectory/Kernel/Init.m as it always returns ...
3
votes
1answer
361 views

Getting data using webscraping (Obtaining links from search engine with redirect)

In the past I have used the function Import in relation with "Hyperlink" to collect hyperlinks and data from the internet. For ...
2
votes
1answer
303 views

Uploading an image to a CloudDeploy Wolfram Language web app

I want to do image processing using the Wolfram Language on a web app hosted on the Wolfram Cloud. The user must be able to upload a PNG image to the Wolfram Cloud, how can I set this up? I've ...
2
votes
1answer
311 views

How to call Imgur API

My image host returns a link starting with http://. It seems this format is not supported in SE anymore. To avoid this, I want to change the image host to Imgur. ...