Questions tagged [web-access]
Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.
250
questions
6
votes
1
answer
184
views
Make Mathematica input things on a website and take the data out
So I found a website that calculates polynomials that I need it intakes a word of a's and b's say aabba and outputs a polynomial. Here is the website: http://www.math.stonybrook.edu/~moira/NewApps/...
- 635
0
votes
1
answer
53
views
HTTPRequest result confusion
everyone!
I've been using Mathematica to progress through the problems of codeabbey.com, and I've picked up the first of "interactive" problems (Problem #176 - Say 100), which is sort of an ...
3
votes
2
answers
66
views
Character encoding of the output from URLExecute
Bug persisting through 13.0.1
I'm using URLExecute to fetch some data from the arXiv preprint server
but it seems that the encoding of the text I get is not ...
- 3,312
1
vote
2
answers
260
views
Getting stock quotes from Yahoo Finance API in Mathematica
I have obtained a Yahoo Finance API key and would like to try it out by making a simple request from within Mathematica. Their tutorial page has some nice little example code snippets in various ...
- 123
0
votes
0
answers
33
views
Is it possible to run SageCellServer inside Mathematica?
I came across this website https://sagecell.sagemath.org/
which can evaluate any SageMath code online.
I wondered if I could access this website (in the background) from a Mathematica notebook, run a ...
- 1,050
0
votes
0
answers
65
views
WebSearch Stopped Working
I've been using 'WebSearch' and 'WebImageSearch' quite a lot recently to sample images for my work. However, these functions suddenly stopped working. When I evaluate, for example,
...
- 4,228
0
votes
0
answers
44
views
scrape App Store web data
I want to scrape the privacy information of an app on the web store page.
For example, the Tiktok app information looks like this:
tiktok on us app store
I found that it's easy to scrape that ...
- 591
0
votes
0
answers
58
views
How to use WebExecute to check the checkbox and to select the item in dropdown list
I use WebExecute to find the "vacuum cleaner" in amazon.com then select in the "Home Vacuum Feature" only the type having the "HEPA filtration", then sort the price from ...
- 125
0
votes
0
answers
36
views
HTTP response: Unauthorized; unless the firefox browser is refreshed
I'm trying to get data with 'URLExecute' from the url below. It's giving "Resource not found". While 'URLRead' is giving
URLRead["https://www.nseindia.com/api/chart-databyindex?index=...
- 91
2
votes
1
answer
61
views
Get page source from active web session?
WebExecute allows to control a web browser through Mathematica. There are various actions one can perform programmatically through this function. However, I could ...
- 11.5k
1
vote
2
answers
74
views
Converting HTML Tables derived from URL's on the Web to Datasets 211112 Revisited
In a previous question [211112] that deals with importing data from a
Wikipedia table directly into a Dataset.
For the following data:
...
- 1,496
1
vote
0
answers
210
views
How to stream NSE option chain data into mathematica
I'm trying to stream option chain data into mathematica from the
url: https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY
...
- 91
1
vote
0
answers
36
views
How do I get access to the element which is inside a document in iframe?
If the element is not inside an iframe then the following works:
...
- 8,217
1
vote
0
answers
203
views
How to check the status of browser after every WebExecute command?
When executing commands using WebExecute on the $CurrentWebSession like follows:
...
- 8,217
2
votes
0
answers
111
views
How to select from a dropdown list in WebExecute?
After locating the dropdown using the XPath as follows:
element=First@WebExecute["LocateElements"->"XPath"->"..."];
How do I ...
- 8,217
2
votes
0
answers
105
views
BlockchainBlockData request limit exceeded
I'm trying to replicate this Blog entry on Bitcoin queries:
https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/
However, I don't get very far ...
- 591
6
votes
1
answer
275
views
How to continue with my Wolfram cloud basic?
I have this simple question but failed to find an answer on the internet.
So far (over a course of 6 months or so) I am using a free account called: Wolfram cloud basic. I use this to make some 3D ...
- 61
6
votes
2
answers
204
views
Is there a CryptocurrencyData alternative to FinancialData?
What's the easiest way to ingest time series data on cryptocurrencies into Mathematica?
I'm looking for up-to-date price, volume, and other properties for both major coins and altcoins.
- 3,679
1
vote
1
answer
95
views
interact with web pages with many hyperlinks and search for a specific name or phrase How?
I have this code that shows many hyperlinks
links = Import[
"http://www.liceobrainstorm.cl/wp-content/uploads/2020/07/",
"Hyperlinks"]
...
- 480
13
votes
1
answer
278
views
Easily download SE posts as notebooks?
Is there a common method of grabbing an SE post or answer here as a formatted notebook so that you can immediately run it, instead of copy-pasting the code blocks?
I've heard SSE uses MarkdownSharp ...
- 30.4k
2
votes
0
answers
60
views
how to get world-wide COVID-19 data in a correct form
I have the following code for retrieving world-wide COVID-19 data.
...
- 4,391
2
votes
1
answer
146
views
How to import a google sheets file that requires account login?
How do I import a google sheets file that is in my google drive?
I tried the following:
...
- 8,217
1
vote
0
answers
46
views
Can Google Earth images be imported?
I've searched for code and found nothing. Is this a server handshake problem?
Below, GeoGraphics executes properly on wikimapia.org, but WebImage doesn't open earth.google.com to a coordinate.
...
- 573
4
votes
1
answer
130
views
Finding Vaccine Appointments in MA
Everyday I spend a crazy amount of time clicking though each maze of hyperlinks and forms on this horrendous site: https://vaxfinder.mass.gov/
Is there anyway that Mathematica can help speed up the ...
- 3,679
4
votes
0
answers
54
views
Socket not ready but data is available
Background
I am playing with https://github.com/arnoudbuzing/wolfram-server and I faced problems with requests that are slightly longer. I tried to narrow it down here.
Problem
This could be my ...
Kuba♦
- 132k
0
votes
1
answer
97
views
Import CSV data from internet to mathematica [closed]
This Link has data in CSV format for 2015 and other years. I would like to get the data for 2015 (by clicking on 2015) but the CSV data format has more rows than Excel can handle and therefore the ...
- 4,391
1
vote
2
answers
137
views
How to use Import to extract data from website?
I would like to extract data of surface properties and grain boundary properties for Cu polymorph #0 from the website: http://crystalium.materialsvirtuallab.org using Mathematica.
So far I tried with <...
- 432
3
votes
1
answer
98
views
Get file downloaded from click in WebSession?
I'm downloading a file by clicking a button from chrome with the WebDriver functionality:
...
- 30.4k
5
votes
2
answers
168
views
How to start an incognito web session?
Is this possible to do with StartWebSession, in either with Chrome or Firefox?
- 3,679
0
votes
0
answers
31
views
WebSessionObject from Process
StartWebSession creates a websession which can be controlled with many commands to automate web stuff. It is limited -- what if one wants to include arguments to ...
- 2,688
2
votes
1
answer
163
views
Using java-script code in Mathematica
I have the following HTML file stored on "C:\local.html":
...
- 2,615
4
votes
2
answers
319
views
How to monitor multiple async image downloads?
I have a few thousand image urls that I want to download asynchronously, how can I do that while monitoring progress? I'm really asking for 5 things:
Show dynamic progress in bar
Downloads must be ...
- 30.4k
0
votes
0
answers
31
views
Expanding website file display ---------show files in full format
Many times looking for some material I find websites that seem to use a format that does not allow the file name to be fully visible.
You download the file that you think it is and it turns out that ...
- 309
6
votes
2
answers
274
views
Curl analog with URLRead/Execute?
I have a working curl command:
curl https://myapp.herokuapp.com/doit -F filedata=@path/to/img.jpg
How do I write this in the modern way with ...
- 30.4k
5
votes
1
answer
175
views
How to send keys to the element in focus with WebExecute
Exploring the WebExecute function, I would like to know how can I send key (or TypeElement) to the element in focus. Here is a toy code using Google.
...
- 25.3k
1
vote
1
answer
208
views
URLDownload(and URLSave, Import) "Error while processing content unencoding"
I'm downloading a file of exactly 1000K-size (https://gitee.com/WolframPaclet1/WolframPacletExtreme1/raw/master/Paclets/MXNetResources-WIN64-12.0.37.paclet.601) with ...
- 702
3
votes
1
answer
259
views
How to use WebExecute to retrieve WebElementObject HTML?
I know that I can use WebExecute to retrieve WebElementObject text, as in this example:
...
- 25.3k
3
votes
2
answers
211
views
Bug in WebExecute TypeElement with diacritics characters [Solved]
[Solved in 12.1.1]
When using WebExecute TypeElement, I get $Failed if I have a diacritics ...
- 25.3k
4
votes
1
answer
101
views
Can Delayed FormFunction redirect to itself (with body)
I want a FormFunction to show a history of inputs, by default an empty list. But once the submit is hit, the form should be reloaded with the history shown on top.
...
Kuba♦
- 132k
2
votes
0
answers
192
views
Cannot launch Chrome or Firefox Driver with StartWebSession[]
The Mathematica 12 in my computer cannot launch Chrome or Firefox session correctly. The first error message is
...
- 21
1
vote
1
answer
644
views
Basic API Key authentication not working with URLExecute
I am a newbie with RESTful APIs and have never tried accessing one from inside Mathematica. I am trying to access the Companies House API, in the UK. The following command works fine from the Terminal ...
- 445
5
votes
0
answers
202
views
Decompose website into rectangular grid?
I'd like to take an image of a website and decompose it into a rectangular clustering of dominant pixels like this:
What I've tried so far
There are a few existing webpage segmentation solutions:
...
- 30.4k
3
votes
0
answers
90
views
Make Mathematica aware of private root certificate authority
I have a private root certificate authority and when I use functions like URLRead[] to connect to a local HTTPS endpoint, I have to add ...
- 1,019
4
votes
2
answers
448
views
SSL errors when using URLFetch or URLSave in versions 8 or 9
It seems that with Mathematica 8 and 9 it is not possible anymore to
download files from SSL encrypted urls. In my case (Fedora 30) on Mathematica 8
(it has no ...
- 3,312
2
votes
1
answer
714
views
URLSave in Mathematica 10 and 11 on Linux [closed]
It seems that URLSave doesn't work as expected with https links in Mathematica 10.4
on Linux. At least in my case (Fedora 30) for
...
- 3,312
2
votes
1
answer
214
views
Convert Plot to Base64 [closed]
My goal is to get a Base64 string directly from a plot (or any Graphics object) in Mathematica such that, when using a Base64 decoder online, such as this one, I ...
- 4,228
5
votes
1
answer
196
views
Using WebExecute to automatically download data
With WebExcute, I have been able to get access to a particular url https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx
However, in this url, there is a quite a number of stock code that ...
- 187
4
votes
0
answers
194
views
Privacy in web-queries [closed]
The question What information is sent to Wolfram Research when loading or working with Mathematica? describes the information sent to Wolfram's servers in routine Mathematica usage (Documentation ...
- 49
3
votes
0
answers
182
views
How to use Mathematica in Chrome extension?
Now that the wolfram engine is free (for developers), I'd like to know how to use it within a chrome extension. It's easy to make a chrome extension, which is essentially just a JSON file with some ...
- 30.4k
0
votes
0
answers
78
views
Yahoo groups is going away: how can I download the old messages?
It looks like all the old email messages from my group are stored in web pages with URLs that look like this:
...
- 65.7k