Questions tagged [web-access]

Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.

Filter by
Sorted by
Tagged with
7
votes
1answer
510 views

Import the table from an html website

I would like to import the data (table) from this website: http://dr.ittf.com/ittf_ranking/ittf_team_ranking.html I don't know how to import and automatically change the date/year. Let's say, I want ...
1
vote
0answers
69 views

Deal with AJAX when importing websites

I'm currently planning to create a small package to enable Mathematica to manipulate WeChat, a extremely famous chatting software in China. The first step is to get the QRCode on the website, but the ...
3
votes
0answers
251 views

URLFetch parameters interfere with POST

(orig: URLFetch Multipart POST fails to post Body?) Update 2: So after much weeping and gnashing of teeth Mathematica deemed me worthy of figuring out (some of) what was going on. It turns out the ...
14
votes
2answers
533 views

Does Mathematica support deep links?

So GitHub sent me on quite the journey, after it opened its desktop app from my browser. Copying the link I found something looking like: "github-mac://...". I ...
2
votes
0answers
24 views

URLFetch fails where import doesn't [duplicate]

So as part of breaking down how to write a service connection I need to use URLFetch as opposed to URLRead. Unfortunately this ...
3
votes
1answer
113 views

Importing from a web page that seems to expect specific browsers

I've successfully and fruitfully imported data from web sites before, but now seem to have a particularly troublesome site that I would like to access through Mathematica on my MacBook. ...
3
votes
1answer
555 views

Can't get a POST URL working with HTTPRequest [duplicate]

I'm puzzled by the syntax of building a URL with HTTPRequest. I can build a URL POST request with postman and that works perfectly. The HTTP code that is build with Postman is: POST /rest/login-...
5
votes
1answer
1k views

How to parse JSON

please forgive me if this is obvious, but I have some JSON: ...
5
votes
1answer
201 views

Decoding GZIP encoded Body, BodyBytes (ByteArray) and BodyBytesArray from URLRead

When calling the Stack Exchange API v2.2 using the new (Mma11) function URLRead we get a GZIP encoded Body in alternative formats: ...
1
vote
0answers
36 views

Has anyone successfully connected to Google Contacts using ServiceConnect in V11?

When I call: googlecontacts = ServiceConnect["GoogleContacts"] The "Wolfram Connector" widget pops up and I click on "Sign in to Google Contacts" which directs me ...
7
votes
1answer
349 views

curl in Mathematica, google link-shortener

I want to automatically get short links equivalents for things I CloudDeploy. I'd appreciate some tips on integrating command line workflow below into Mathematica ...
8
votes
1answer
292 views

What is the equivalent of curl?

The URL related functions have confused me yet again (I have v11.1)! What is the Mathematica equivalent for the following generic curl command? ...
7
votes
1answer
210 views

How to get StatusCode without fetching Body/Content in the background

I only need to know whether a specific call returns 200 but: ...
4
votes
1answer
295 views

Is it possible to grab all the links at once on a webpage?

I'm working with some webscraping using Mathematica and have run into a question: is it possible to grab all the links on a webpage that scrolls down? The page I am trying to scrape is this: "http://...
3
votes
1answer
205 views

Parsing JavaScript Script from XML Trees

I've been trying to parse some script written in JavaScript while scraping a website, and am a little confused what functions are needed to get this done in Mathematica. I imported the data using <...
8
votes
1answer
193 views

Authentication dialog for URLDownload / URLSave

Bug introduced in 11.0 and persists through 11.3 The original problem is was fixed in V11.2 but since then it returns File[path/to/file/that/does/not/exits] ...
6
votes
1answer
252 views

How to upload an image to the Stack Exchange Imgur account?

This is not a duplicate of How to call Imgur API. I tried to upload to imgur using its API, but (as my friends test) the API of imgur cannot be called in Chinese mainland. Then I found that ...
3
votes
1answer
115 views

Import::fmterr URLExecute wont import JSON

I'm trying to perform a multiple element GET request from an alfresco database of elements. The way we have it set up is if a GET request is made with a body, it will return the elements listed in the ...
2
votes
2answers
340 views

Using URLFetch/URLExecute to post to an internal server

I've been looking up how to do this for a while and I haven't seen any thorough explanations, just hints. Is there like a secret repository of clearly defined Mathematica code somewhere that I cant ...
2
votes
1answer
308 views

How to call Imgur API

My image host returns a link starting with http://. It seems this format is not supported in SE anymore. To avoid this, I want to change the image host to Imgur. ...
3
votes
1answer
59 views

Divergent import behavior on URL and HTTPRequest

I'm playing with the IMSLP API (such as it is) and was running it through my generalized API interface which formats calls to HTTPRequest for reasons of flexibility,...
8
votes
1answer
161 views

Inconsistent Authorization Dialog (401) handling

Bug introduced in V11 and fixed in 11.3 [CASE:3855554] was created [...] It does appear that the URLRead is not behaving correctly. I have forwarded an incident report to our developers with ...
6
votes
1answer
87 views

Inconsistent HTTPResponse handling

It seems that sources of URL* related features are not initialized properly which leads to problems in many cases. Please follow those steps to reproduce the ...
1
vote
1answer
108 views

How do I use Mathematica with a web-based tool?

I use Mathematica to generate lists of genes from Saccharomyces Cerevisiae that I then throw into the GO enrichment analysis tool here: http://pantherdb.org/webservices/go/overrep.jsp. I know how to ...
3
votes
1answer
86 views

URLDownload aborts prematurely

So I've been trying to automate the download and build of various large packages for hooking Mathematica into extensions but the downloads continually finish prematurely. Trying to get the Psi4 ...
6
votes
3answers
267 views

Possible bug with URLRead

Version 11 introduces URLRead (and others) to replace URLFetch, but I have run into a strange problem while trying to switch to ...
6
votes
1answer
352 views

Import information from a web URL that requires a username and password

I wish to use: Import[link, "Plaintext"] to get the plain text on a web page - however the webpage is on an internal network and requires a username and password ...
3
votes
1answer
47 views

Import file from website using process indiciator

Would it be possible to have some sort of progress indicator for downloading files? For example if we would like to import: ...
4
votes
1answer
124 views

URLSave and bitbucket

I'm trying to download a file hosted on bitbucket via Mathematica. The direct link https://bitbucket.org/feynmanIntegrals/fire/downloads/FIRE5.2.tar.gz works fine with Firefox and wget. However, ...
4
votes
0answers
233 views

How to convert a URLFetch to a URLRead or URLExecute?

Consider the following function: ...
2
votes
1answer
188 views

How to omit the authentification dialog during URLRead?

When I use URLRead Mathematica will popup a window like: Actually I have given the parameters Password and ...
1
vote
1answer
120 views

How to create string from {“jsonrpc” -> “2.0”, “method” -> method, “params” -> {“filter” -> “{}”}}

Consider the following function: ...
4
votes
2answers
273 views

How to call Picture Bed API

I want to call this Picture bed API to upload my picture and return its link.I have translated the official instructional the page into English.And there is a example succeed to use it with Python. I ...
7
votes
1answer
604 views

Accessing the Twitter API using Mathematica 11

The built-in social media functions are limited, so I'm using Twitter's web API for my work. The following function worked for me when I was using versions of Mathematica earlier than 10.3. ...
10
votes
0answers
107 views

URL functions overload

All these functions seem capable of doing the same things: Import, URLFetch, URLSave, ...
-5
votes
1answer
55 views

Passing the name of an url to ma [closed]

I have the following command How to circonvent this weird comportment. Perhaps, it is not possible to open a pdf file from mathematica. But this is not the full problem.
3
votes
0answers
112 views

Use Mathematica inside web app?

My python code running in a Flask web application needs to return results from Mathematica, what is the best way to achieve this? Are there any examples of this having been done successfully?
6
votes
1answer
209 views

How to use URLDownload if the url doesn't have the correct filetype extension?

I'm using the new (in V11) function UrlDownload[] I have a url I'd like to save to a local file, but the url is missing a (or has the incorrect) filetype extension....
12
votes
0answers
189 views

Is it possible to search the web by image from Mathematica?

Is it possible to perform a web search based on an image input? There are several services that provide this: Google, Bing, TinEye, and maybe others. I am not sure which can be accessed ...
6
votes
1answer
199 views

How to use the new function FindCookies?

This is a further question of this post. Actually this code from SqRoots.And this code has completed the task. (I assume you are using Mozilla Firefox and MMA version 11.0) ...
22
votes
1answer
800 views

Can Mathematica act like a web server?

I have a Mathematica program that modifies images that I would like to make available for other people to use through the web. The input is only one parameter. So, for example, I could run it as a ...
7
votes
0answers
573 views

How to crawl some website needed username and password

In version 11,if we want to crawl public website like wiki,we can do it like this: URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"] But I don't know how ...
1
vote
0answers
166 views

Make Mathematica input stuffs on websites

Main Question I'm currently making a web crawler, but I'll need to deal with login windows sometimes. All simple web crawlers created with Mathematica I've seen before cannot deal with input boxes or ...
3
votes
0answers
54 views

Ideas on source to learn the ins and outs of all Web Operations?

Since beginning to learn Mathematica a few months ago, I have been amazed by its capabilities. I have recently come across basic web operations such as URLFetch and I am interested in learning more. ...
7
votes
0answers
125 views

How to `URLEncode` spaces properly?

URLEncode encodes spaces as plus symbols, but this isn't what we want: The encoding used by default is based on a very early version of the general URI percent-...
1
vote
0answers
96 views

URLFetch meet the problem of 502

I'm using Mathematica10.0.2 on Win7 to write a crawler. Here is a piece of my code: ...
0
votes
1answer
171 views

How to extract an answer on Mathematica.SE as an image [duplicate]

When I try to get this answer from Mathematica.SE by evaluating Import["https://mathematica.stackexchange.com/a/95725/21532"] I just get some text without any ...
7
votes
2answers
351 views

Web Browsing Automation: how to expand all comments under a blog post?

I wish to expand all the comments under a LiveJournal post for the purposes of saving the complete thread of comments. According to LiveJournal's FAQ when a post gets more than 50 comments they are ...
2
votes
0answers
117 views

Web crawler: want to ignore login windows [closed]

I'm trying to make web crawler with Mathematica. ...
4
votes
1answer
158 views

How do I use Mathematica to upload to Clyp?

Clyp is a service for hosting sound files that (as of now) doesn't require registration. I'm wondering if it's possible to write a function that'll upload a Sound ...