Questions tagged [web-access]
Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.
266 questions
7
votes
2
answers
516
views
"Accept" header in HTTP request for DOI Content Negotiation
I am trying to get the .bib info from a DOI, following the guide here.
Basically, I wish to run
...
- 37.4k
11
votes
1
answer
602
views
Using URLExecute to login to website
One of my students is doing summer research for a local company to help analyze some of their data. Trouble is, the web interface they provide for downloading the data only allows one to download one ...
- 71.2k
2
votes
0
answers
140
views
What could cause an "Uncaught Throw[$Failed,CURLLink`Utilities`Exception] returned to top level." error message?
I am running a program in Mathematica that uses web browser automation, and is also connected to my email account. The program runs fine for a while, but after a few hours, I end up getting the ...
- 1,447
0
votes
0
answers
42
views
Verify that a document matches a URL
Writing:
Quiet[Length[Import["http://www.africau.edu/images/default/sample.pdf"]]]
I get:
2
while writing:
...
- 4,462
6
votes
1
answer
532
views
How to click a button to upload local file with WebExecute?
A webpage has a "browse" button to open the filesystem to select a file and upload it, in html it looks like this:
...
- 9,831
2
votes
0
answers
41
views
How to overcome the threshold 500 ExternalLinks for downloading of the WikipediaData?
The command, for instance,
WikipediaData["Global warming", "ExternalLinks"]
gives me 500 links only, as so as
...
- 1,782
2
votes
0
answers
121
views
How to use EmbedCode with CloudDeploy correctly?
I ran EmbedCode[CloudDeploy[Manipulate[Plot[Sin[x + c], {x, -6, 6}], {c, -1, 1}]]] based on this instruction and obtained the html code for embedding as below:-
<...
- 3,509
3
votes
2
answers
270
views
Number of line in website [closed]
I want to download the content of the website(contains text) and only a few lines from the content (from a specific number of the line up to the last line minus specific offset). Unfortunately, I do ...
- 4,341
1
vote
0
answers
37
views
WebImageSearch for images of high resolution?
Is there a way to specify a minimal image dimension when using WebImageSearch? Since it costs money I don't want to do a large search and then filter the results.
...
- 31.8k
4
votes
2
answers
208
views
WebExecute timeout
There isn't much tuning freedom available in the WebExecute method. If a page takes too long to load you only see a spinner or a blank for CapturePage. Does someone ...
- 9,831
2
votes
1
answer
104
views
How can I deal with a 502 error? [closed]
I am working on a program that interfaces with a web API. Mathematica runs perfectly fine for a few hours and then, after a while, will sometimes return:
The request to URL
https://api.polygon.io/...
- 1,447
8
votes
1
answer
2k
views
Download Youtube Videos
Is there a way to display or download youtube videos in Mathematica.
Youtube is HTML5 enabled so I am hopeful there might be an easy way.
CommunityBot
- 1
3
votes
1
answer
783
views
How can I convert $\TeX$ image to Wolfram expression via MathPix
Mathpix is a tool which can covert $\TeX$ image to $\TeX$ expression, they also say they can return a string which is compatible with the Wolfram Alpha engine if possible.
I found their documentation ...
- 436
10
votes
1
answer
506
views
Import data from dynamically generated webpage
I'm trying to get data from a web-page
data=Import["https://stats.nba.com/team/1610612738/traditional/", "Data"]
It returns only a textual information. ...
- 13.2k
2
votes
1
answer
309
views
How to get URL request response in Mathematica?
I am considering the following link:
https://myimmitracker.com/en/ca/trackers/consolidated-e-apr-tracker-express-entry-permanent-residency-application
It produces the following page with a table in ...
1
vote
0
answers
49
views
Chrome drive broken? [closed]
I tried this external session, as Import["Url"] is sometimes blocked by websites. But it is down now, and I really don't know why. My computer is Dell XPS 13 with ...
- 2,031
1
vote
2
answers
96
views
What info is exchanged when using Import[url]?
When using Import with a url what information is broadcasted/sent to the receiving end of the request and is it possible to monitor/modify that information?
1
vote
0
answers
82
views
How to access image from ethernet camera [duplicate]
I have a Edimax IC-3140W ethernet security camera. It acts as a web host where the video feed can be viewed in a browser. Navigating to its ip address http://ip.ip.ip.ip, it will ask for a password ...
- 2,590
11
votes
2
answers
275
views
How to show Plots or Graphics in mini MMA HTTP Server?
I learned some codes to make a mini HTTP server in MMA way here.
...
- 138k
0
votes
0
answers
132
views
Scraping Images that are Directly Embedded in a Webpage or Dynamic?
I need to scrape URLs for photo albums like Flickr and Facebook for images from a certain event.
I'm trying to use Import["link", "Images"], as shown at
https://...
- 138k
3
votes
0
answers
150
views
How to connect to ncbi databases for text search? MathIOmica authoring instructions now do not work [closed]
Mias's suplementary material in the notebook MmaBioinfo_Ch4_Databases contains a code for search the snp related to autism
...
- 138k
2
votes
0
answers
100
views
create interactive local site with backgroud mathematica calculations
This may be a bit of a weird question but I cannot figure out if this is possible. I need to write a program at work that can be used interactively. By that I mean, it will have several parameters ...
- 728
3
votes
1
answer
193
views
Importing data from an https server
When trying to import data from an https request I get the following error message.
...
- 138k
6
votes
1
answer
422
views
webscraping: Wolfram notebook vs browser
In the Netherlands there is a website Funda.nl There you can find all houses for sale or for rent. I used webscraping as a tool to collect data for analyse-purpose. The script I used is like:
...
- 62.3k
6
votes
1
answer
239
views
Export Data from several tabs in a website
Here is the website from the official federation:
http://results.ittf.link/index.php?option=com_fabrik&view=list&listid=36&Itemid=158
I want to export the Data (the table). There are ...
- 62.3k
5
votes
1
answer
241
views
How can I tally users on the Wolfram Challenges leaderboard?
Wolfram Challenges has a leaderboard that features users that have completed the challenges in the 'best' way, according to specified criteria like efficiency or shortness of code.
Can I use ...
- 47.1k
5
votes
2
answers
616
views
Mathematica Import functions fails for data download from website
I am trying to download some data from a website that uses "java enabling" to tell me that I cannot download the data. Using
Import[ "website...", "Data"]
I get
...
- 71.2k
12
votes
1
answer
459
views
A tour of AWSLink?
Seems like there are a lot of useful functions in this undocumented package AWSLink:
Any developers on here that can demo it a bit?
- 31.8k
6
votes
0
answers
180
views
Connecting to Twitter API Manually in Mathematica to Pull Down All Followers of a Specified User (e.g. More than 5,000)
I have been going through old StackExchange threads to see how to potentially use Mathematica's built-in connection/authentication to the Twitter API to connect to it manually, and access some ...
- 81
3
votes
2
answers
153
views
Export an image as pdf that acts as a link mathematica
Anyone knows if it is possible to export an image as a pdf and this image should act as a link to a website when one clicks on it form within a pdf viewer ?
- 37.4k
8
votes
1
answer
926
views
Programmatic browser interaction: How to click on web-page and get new URL and list of links
In Mathematica it's easy to Import data from the web and perform all sort of Web Operations. But the later doesn't allow to simulate a more interactive browser ...
- 37.4k
2
votes
1
answer
112
views
Select the right XMLelement
I scrape data from a public website about houses for rent. All pages have the same structure. But the pages can differ in the amount of information.
Below I give two examples:
page/house 1:
...
- 70.2k
4
votes
1
answer
1k
views
Getting data using webscraping (Obtaining links from search engine with redirect)
In the past I have used the function Import in relation with "Hyperlink" to collect hyperlinks and data from the internet.
For ...
- 37.4k
1
vote
0
answers
157
views
Download YouTube files using Mathematica
I am not that familiar with other languages, but I know Mathematica is powerful enough to excess webpage data (youtube copyright free database). I want to download sound effects from YouTube, but I ...
- 47.1k
4
votes
0
answers
76
views
Possible memory leak in URLRead? (MMA v11.3)
I was writing a API with a lot of http requests, when I noticed Mathematica was slowly using more and more memory, and after some debugging I think the culprit is ...
- 841
7
votes
1
answer
221
views
Does Dropbox sync smartly notebook files? [duplicate]
It is said that Dropbox employs a Delta Sync technology where only the modified part of a file is updated through the internet. I was just wondering if it holds for notebook files.
I sometimes ...
- 113k
5
votes
2
answers
112
views
Pulling sections from a webpage
I'm trying to get specific sections from a webpage using Mathematica. I've started by importing the webpage, and getting the section "headers" that I want:
...
- 71.2k
1
vote
0
answers
75
views
How to I start viewing a YouTube Video at, say, 7 minutes in? [closed]
It is very easy to load and run a YouTube video, for example via:
...
- 331
7
votes
1
answer
161
views
How to cache results of WebImageSearch?
Every call to WebImageSearch costs money (wolfram service credits), thus, I would like to page through results without having to redownload them.
Given the ...
- 31.8k
19
votes
1
answer
317
views
How can I get every function a user has made on StackExchange?
Yesterday I was looking at one of Henrik Schumacher's answers and realizing that it'd be useful to some of these functions more accessible for reuse. Copy-and-paste is nice, but better would be a ...
- 47.1k
1
vote
1
answer
438
views
Is it possible to embed HTML webpage in a Mathematica notebook?
I am trying to display a webpage as an iframe in Mathematica.
The only option I have found is with EmbededHTML, but I can't get it to work for me. Example:
<...
- 138k
4
votes
1
answer
255
views
HTTPRequest with multiple parameters
Want to use the API from this site.
I'm having trouble using the same parameters.
But I can do this with curl as shown on the site:
...
- 138k
3
votes
2
answers
177
views
Importing from an API that has non UTF-8 strings
I am new to both Mathematica and API's. I am trying to access a dictionary of player-ids to names from the website datdota.com, but some of them have chinese characters.
relevant code:
...
8
votes
1
answer
178
views
Inconsistent Authorization Dialog (401) handling
Bug introduced in V11 and fixed in 11.3
[CASE:3855554] was created
[...] It does appear that the URLRead is not behaving correctly. I have forwarded an incident report to our developers with ...
- 138k
4
votes
1
answer
101
views
URLSubmit handling of query parameters
Bug introduced in 11.2 or earlier and persisting through 11.3
[CASE:4007421] confirmed
myApi = CloudDeploy@APIFunction[{"i" -> "Real"}, #i &];
Why does ...
- 138k
9
votes
1
answer
251
views
Authentication dialog for URLDownload / URLSave
Bug introduced in 11.0 and persists through 11.3
The original problem is was fixed in V11.2 but since then it returns File[path/to/file/that/does/not/exits] instead ...
CommunityBot
- 1
9
votes
1
answer
133
views
HTTPRequest syntax problem
Bug introduced in V11.0 and persisting through 11.3
[CASE:3981157] confirmed
...
- 138k
5
votes
1
answer
185
views
Cloud CDF and HTTPRedirect
Cross posted on community.wolfram.com
Question and requirements
Is it possible to HTTPRedirect from a Cloud CDF via e.g. Button...
- 146
2
votes
0
answers
2k
views
Execute `sendPhoto` to telegram using telegram-bot api
I was create telegram-bot using Mathematica. All the possibilities of telegram api can be found here. To send images, you can use method sendPhoto. To upload image ...
- 819
1
vote
0
answers
80
views
Truncated tweets texts
I use the Twitter command
twitter = ServiceConnect["Twitter", "New"]
listTws = twitter["TweetSearch", "Query" -> "Trump", MaxItems -> 10];
For example, the ...
- 1,688