Questions tagged [web-access]
Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.
247
questions
0
votes
0answers
29 views
Is it possible to run SageCellServer inside Mathematica?
I came across this website https://sagecell.sagemath.org/
which can evaluate any SageMath code online.
I wondered if I could access this website (in the background) from a Mathematica notebook, run a ...
0
votes
0answers
57 views
WebSearch Stopped Working
I've been using 'WebSearch' and 'WebImageSearch' quite a lot recently to sample images for my work. However, these functions suddenly stopped working. When I evaluate, for example,
...
0
votes
0answers
38 views
scrape App Store web data
I want to scrape the privacy information of an app on the web store page.
For example, the Tiktok app information looks like this:
tiktok on us app store
I found that it's easy to scrape that ...
0
votes
0answers
49 views
How to use WebExecute to check the checkbox and to select the item in dropdown list
I use WebExecute to find the "vacuum cleaner" in amazon.com then select in the "Home Vacuum Feature" only the type having the "HEPA filtration", then sort the price from ...
0
votes
0answers
32 views
HTTP response: Unauthorized; unless the firefox browser is refreshed
I'm trying to get data with 'URLExecute' from the url below. It's giving "Resource not found". While 'URLRead' is giving
URLRead["https://www.nseindia.com/api/chart-databyindex?index=...
2
votes
1answer
51 views
Get page source from active web session?
WebExecute allows to control a web browser through Mathematica. There are various actions one can perform programmatically through this function. However, I could ...
1
vote
2answers
66 views
Converting HTML Tables derived from URL's on the Web to Datasets 211112 Revisited
In a previous question [211112] that deals with importing data from a
Wikipedia table directly into a Dataset.
For the following data:
...
1
vote
0answers
116 views
How to stream NSE option chain data into mathematica
I'm trying to stream option chain data into mathematica from the
url: https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY
...
1
vote
0answers
28 views
How do I get access to the element which is inside a document in iframe?
If the element is not inside an iframe then the following works:
...
1
vote
0answers
182 views
How to check the status of browser after every WebExecute command?
When executing commands using WebExecute on the $CurrentWebSession like follows:
...
2
votes
0answers
82 views
How to select from a dropdown list in WebExecute?
After locating the dropdown using the XPath as follows:
element=First@WebExecute["LocateElements"->"XPath"->"..."];
How do I ...
2
votes
0answers
94 views
BlockchainBlockData request limit exceeded
I'm trying to replicate this Blog entry on Bitcoin queries:
https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/
However, I don't get very far ...
6
votes
1answer
180 views
How to continue with my Wolfram cloud basic?
I have this simple question but failed to find an answer on the internet.
So far (over a course of 6 months or so) I am using a free account called: Wolfram cloud basic. I use this to make some 3D ...
6
votes
2answers
161 views
Is there a CryptocurrencyData alternative to FinancialData?
What's the easiest way to ingest time series data on cryptocurrencies into Mathematica?
I'm looking for up-to-date price, volume, and other properties for both major coins and altcoins.
1
vote
1answer
92 views
interact with web pages with many hyperlinks and search for a specific name or phrase How?
I have this code that shows many hyperlinks
links = Import[
"http://www.liceobrainstorm.cl/wp-content/uploads/2020/07/",
"Hyperlinks"]
...
13
votes
1answer
278 views
Easily download SE posts as notebooks?
Is there a common method of grabbing an SE post or answer here as a formatted notebook so that you can immediately run it, instead of copy-pasting the code blocks?
I've heard SSE uses MarkdownSharp ...
2
votes
0answers
58 views
how to get world-wide COVID-19 data in a correct form
I have the following code for retrieving world-wide COVID-19 data.
...
2
votes
1answer
96 views
How to import a google sheets file that requires account login?
How do I import a google sheets file that is in my google drive?
I tried the following:
...
1
vote
0answers
39 views
Can Google Earth images be imported?
I've searched for code and found nothing. Is this a server handshake problem?
Below, GeoGraphics executes properly on wikimapia.org, but WebImage doesn't open earth.google.com to a coordinate.
...
0
votes
0answers
28 views
Accessing a web page
Hi I intend to import some data from this website but I cannot pass the validation process (of course I have the username and password). It seems that I have to give more inputs given the site ...
4
votes
1answer
97 views
Finding Vaccine Appointments in MA
Everyday I spend a crazy amount of time clicking though each maze of hyperlinks and forms on this horrendous site: https://vaxfinder.mass.gov/
Is there anyway that Mathematica can help speed up the ...
4
votes
0answers
48 views
Socket not ready but data is available
Background
I am playing with https://github.com/arnoudbuzing/wolfram-server and I faced problems with requests that are slightly longer. I tried to narrow it down here.
Problem
This could be my ...
0
votes
1answer
86 views
Import CSV data from internet to mathematica [closed]
This Link has data in CSV format for 2015 and other years. I would like to get the data for 2015 (by clicking on 2015) but the CSV data format has more rows than Excel can handle and therefore the ...
1
vote
2answers
114 views
How to use Import to extract data from website?
I would like to extract data of surface properties and grain boundary properties for Cu polymorph #0 from the website: http://crystalium.materialsvirtuallab.org using Mathematica.
So far I tried with <...
3
votes
1answer
86 views
Get file downloaded from click in WebSession?
I'm downloading a file by clicking a button from chrome with the WebDriver functionality:
...
5
votes
2answers
155 views
How to start an incognito web session?
Is this possible to do with StartWebSession, in either with Chrome or Firefox?
0
votes
0answers
30 views
WebSessionObject from Process
StartWebSession creates a websession which can be controlled with many commands to automate web stuff. It is limited -- what if one wants to include arguments to ...
2
votes
1answer
127 views
4
votes
2answers
305 views
How to monitor multiple async image downloads?
I have a few thousand image urls that I want to download asynchronously, how can I do that while monitoring progress? I'm really asking for 5 things:
Show dynamic progress in bar
Downloads must be ...
0
votes
0answers
31 views
Expanding website file display ---------show files in full format
Many times looking for some material I find websites that seem to use a format that does not allow the file name to be fully visible.
You download the file that you think it is and it turns out that ...
6
votes
2answers
249 views
Curl analog with URLRead/Execute?
I have a working curl command:
curl https://myapp.herokuapp.com/doit -F filedata=@path/to/img.jpg
How do I write this in the modern way with ...
5
votes
1answer
134 views
How to send keys to the element in focus with WebExecute
Exploring the WebExecute function, I would like to know how can I send key (or TypeElement) to the element in focus. Here is a toy code using Google.
...
1
vote
1answer
189 views
URLDownload(and URLSave, Import) "Error while processing content unencoding"
I'm downloading a file of exactly 1000K-size (https://gitee.com/WolframPaclet1/WolframPacletExtreme1/raw/master/Paclets/MXNetResources-WIN64-12.0.37.paclet.601) with ...
3
votes
1answer
210 views
How to use WebExecute to retrieve WebElementObject HTML?
I know that I can use WebExecute to retrieve WebElementObject text, as in this example:
...
3
votes
2answers
193 views
Bug in WebExecute TypeElement with diacritics characters [Solved]
[Solved in 12.1.1]
When using WebExecute TypeElement, I get $Failed if I have a diacritics ...
4
votes
1answer
99 views
Can Delayed FormFunction redirect to itself (with body)
I want a FormFunction to show a history of inputs, by default an empty list. But once the submit is hit, the form should be reloaded with the history shown on top.
...
2
votes
0answers
165 views
Cannot launch Chrome or Firefox Driver with StartWebSession[]
The Mathematica 12 in my computer cannot launch Chrome or Firefox session correctly. The first error message is
...
1
vote
1answer
539 views
Basic API Key authentication not working with URLExecute
I am a newbie with RESTful APIs and have never tried accessing one from inside Mathematica. I am trying to access the Companies House API, in the UK. The following command works fine from the Terminal ...
5
votes
0answers
191 views
Decompose website into rectangular grid?
I'd like to take an image of a website and decompose it into a rectangular clustering of dominant pixels like this:
What I've tried so far
There are a few existing webpage segmentation solutions:
...
3
votes
0answers
87 views
Make Mathematica aware of private root certificate authority
I have a private root certificate authority and when I use functions like URLRead[] to connect to a local HTTPS endpoint, I have to add ...
4
votes
2answers
404 views
SSL errors when using URLFetch or URLSave in versions 8 or 9
It seems that with Mathematica 8 and 9 it is not possible anymore to
download files from SSL encrypted urls. In my case (Fedora 30) on Mathematica 8
(it has no ...
2
votes
1answer
644 views
URLSave in Mathematica 10 and 11 on Linux [closed]
It seems that URLSave doesn't work as expected with https links in Mathematica 10.4
on Linux. At least in my case (Fedora 30) for
...
2
votes
1answer
178 views
Convert Plot to Base64 [closed]
My goal is to get a Base64 string directly from a plot (or any Graphics object) in Mathematica such that, when using a Base64 decoder online, such as this one, I ...
5
votes
1answer
180 views
Using WebExecute to automatically download data
With WebExcute, I have been able to get access to a particular url https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx
However, in this url, there is a quite a number of stock code that ...
4
votes
0answers
192 views
Privacy in web-queries [closed]
The question What information is sent to Wolfram Research when loading or working with Mathematica? describes the information sent to Wolfram's servers in routine Mathematica usage (Documentation ...
3
votes
0answers
174 views
How to use Mathematica in Chrome extension?
Now that the wolfram engine is free (for developers), I'd like to know how to use it within a chrome extension. It's easy to make a chrome extension, which is essentially just a JSON file with some ...
0
votes
0answers
78 views
Yahoo groups is going away: how can I download the old messages?
It looks like all the old email messages from my group are stored in web pages with URLs that look like this:
...
10
votes
1answer
224 views
Mathematica 12: Failure with URLRead
Bug introduced in 12.0
The following HTTP request fails to submit the body under version 12, but works in Mathematica 11:
...
12
votes
1answer
372 views
How to work with ElasticSearch in Mathematica?
Steps to install and run ElasticSearch:
install Java (64-bit): https://www.java.com/en/download/manual.jsp
download ElasticSearch (https://www.elastic.co/start) and extract archive in ...
4
votes
2answers
357 views
Can WebExcute click on a button on a webpage that has been created with Javascript?
In an effort to explore the new WebExecute capabilities in Version 12 I have created a simple algorithm to download some data that are stored on an external server and displayed on a webpage. I am ...