Questions tagged [web-access]
Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.
246
questions
-1
votes
0answers
72 views
Web scraping: Price search and compare prices from different appointed websites
Is it possible to search and compare for the price of a certain product, e.g., "Dầu thực vật Tường An", in different appointed websites, e.g., https://www.sendo.vn, https://www.lazada.vn, ...
0
votes
0answers
26 views
HTTP response: Unauthorized; unless the firefox browser is refreshed
I'm trying to get data with 'URLExecute' from the url below. It's giving "Resource not found". While 'URLRead' is giving
URLRead["https://www.nseindia.com/api/chart-databyindex?index=...
2
votes
1answer
49 views
Get page source from active web session?
WebExecute allows to control a web browser through Mathematica. There are various actions one can perform programmatically through this function. However, I could ...
1
vote
2answers
64 views
Converting HTML Tables derived from URL's on the Web to Datasets 211112 Revisited
In a previous question [211112] that deals with importing data from a
Wikipedia table directly into a Dataset.
For the following data:
...
1
vote
0answers
105 views
How to stream NSE option chain data into mathematica
I'm trying to stream option chain data into mathematica from the
url: https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY
...
1
vote
0answers
26 views
How do I get access to the element which is inside a document in iframe?
If the element is not inside an iframe then the following works:
...
1
vote
0answers
97 views
How to check the status of browser after every WebExecute command?
When executing commands using WebExecute on the $CurrentWebSession like follows:
...
1
vote
0answers
54 views
How to select from a dropdown list in WebExecute?
After locating the dropdown using the XPath as follows:
element=First@WebExecute["LocateElements"->"XPath"->"..."];
How do I ...
2
votes
0answers
90 views
BlockchainBlockData request limit exceeded
I'm trying to replicate this Blog entry on Bitcoin queries:
https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/
However, I don't get very far ...
6
votes
1answer
141 views
How to continue with my Wolfram cloud basic?
I have this simple question but failed to find an answer on the internet.
So far (over a course of 6 months or so) I am using a free account called: Wolfram cloud basic. I use this to make some 3D ...
6
votes
2answers
143 views
Is there a CryptocurrencyData alternative to FinancialData?
What's the easiest way to ingest time series data on cryptocurrencies into Mathematica?
I'm looking for up-to-date price, volume, and other properties for both major coins and altcoins.
1
vote
1answer
88 views
interact with web pages with many hyperlinks and search for a specific name or phrase How?
I have this code that shows many hyperlinks
links = Import[
"http://www.liceobrainstorm.cl/wp-content/uploads/2020/07/",
"Hyperlinks"]
...
13
votes
1answer
272 views
Easily download SE posts as notebooks?
Is there a common method of grabbing an SE post or answer here as a formatted notebook so that you can immediately run it, instead of copy-pasting the code blocks?
I've heard SSE uses MarkdownSharp ...
1
vote
1answer
68 views
How to import a google sheets file that requires account login?
How do I import a google sheets file that is in my google drive?
I tried the following:
...
1
vote
0answers
38 views
Can Google Earth images be imported?
I've searched for code and found nothing. Is this a server handshake problem?
Below, GeoGraphics executes properly on wikimapia.org, but WebImage doesn't open earth.google.com to a coordinate.
...
0
votes
0answers
27 views
Accessing a web page
Hi I intend to import some data from this website but I cannot pass the validation process (of course I have the username and password). It seems that I have to give more inputs given the site ...
4
votes
1answer
95 views
Finding Vaccine Appointments in MA
Everyday I spend a crazy amount of time clicking though each maze of hyperlinks and forms on this horrendous site: https://vaxfinder.mass.gov/
Is there anyway that Mathematica can help speed up the ...
4
votes
0answers
45 views
Socket not ready but data is available
Background
I am playing with https://github.com/arnoudbuzing/wolfram-server and I faced problems with requests that are slightly longer. I tried to narrow it down here.
Problem
This could be my ...
0
votes
1answer
78 views
Import CSV data from internet to mathematica [closed]
This Link has data in CSV format for 2015 and other years. I would like to get the data for 2015 (by clicking on 2015) but the CSV data format has more rows than Excel can handle and therefore the ...
0
votes
0answers
35 views
Chrome keeps changing the default download folder, Can I use WebExecute to override or otherwise convince Chrome to accept my preference?
I have been pleased to find WebExecute for logging into web sites and downloading data that is updated daily from other sources, but Chrome keeps downloading the files to the factory default /download ...
1
vote
2answers
106 views
How to use Import to extract data from website?
I would like to extract data of surface properties and grain boundary properties for Cu polymorph #0 from the website: http://crystalium.materialsvirtuallab.org using Mathematica.
So far I tried with <...
3
votes
1answer
81 views
Get file downloaded from click in WebSession?
I'm downloading a file by clicking a button from chrome with the WebDriver functionality:
...
0
votes
0answers
48 views
Web crawler, where access is needed
I want to make a web crawler and download a bunch of files that are located in different directories on the website. Accessing the files need username and password input as well (the website is https:/...
5
votes
2answers
146 views
How to start an incognito web session?
Is this possible to do with StartWebSession, in either with Chrome or Firefox?
0
votes
0answers
27 views
WebSessionObject from Process
StartWebSession creates a websession which can be controlled with many commands to automate web stuff. It is limited -- what if one wants to include arguments to ...
2
votes
1answer
115 views
4
votes
2answers
294 views
How to monitor multiple async image downloads?
I have a few thousand image urls that I want to download asynchronously, how can I do that while monitoring progress? I'm really asking for 5 things:
Show dynamic progress in bar
Downloads must be ...
0
votes
0answers
31 views
Expanding website file display ---------show files in full format
Many times looking for some material I find websites that seem to use a format that does not allow the file name to be fully visible.
You download the file that you think it is and it turns out that ...
6
votes
2answers
226 views
Curl analog with URLRead/Execute?
I have a working curl command:
curl https://myapp.herokuapp.com/doit -F filedata=@path/to/img.jpg
How do I write this in the modern way with ...
4
votes
1answer
121 views
How to send keys to the element in focus with WebExecute
Exploring the WebExecute function, I would like to know how can I send key (or TypeElement) to the element in focus. Here is a toy code using Google.
...
1
vote
1answer
169 views
URLDownload(and URLSave, Import) "Error while processing content unencoding"
I'm downloading a file of exactly 1000K-size (https://gitee.com/WolframPaclet1/WolframPacletExtreme1/raw/master/Paclets/MXNetResources-WIN64-12.0.37.paclet.601) with ...
3
votes
1answer
187 views
How to use WebExecute to retrieve WebElementObject HTML?
I know that I can use WebExecute to retrieve WebElementObject text, as in this example:
...
3
votes
2answers
184 views
Bug in WebExecute TypeElement with diacritics characters [Solved]
[Solved in 12.1.1]
When using WebExecute TypeElement, I get $Failed if I have a diacritics ...
4
votes
1answer
98 views
Can Delayed FormFunction redirect to itself (with body)
I want a FormFunction to show a history of inputs, by default an empty list. But once the submit is hit, the form should be reloaded with the history shown on top.
...
2
votes
0answers
151 views
Cannot launch Chrome or Firefox Driver with StartWebSession[]
The Mathematica 12 in my computer cannot launch Chrome or Firefox session correctly. The first error message is
...
1
vote
1answer
481 views
Basic API Key authentication not working with URLExecute
I am a newbie with RESTful APIs and have never tried accessing one from inside Mathematica. I am trying to access the Companies House API, in the UK. The following command works fine from the Terminal ...
5
votes
0answers
187 views
Decompose website into rectangular grid?
I'd like to take an image of a website and decompose it into a rectangular clustering of dominant pixels like this:
What I've tried so far
There are a few existing webpage segmentation solutions:
...
8
votes
0answers
184 views
Guide to capturing the complete image of given urls?
I'm trying to use WebImage[] to capture the complete image of a website (yes, it might be very tall). I seem to be having some issues with it:
How to attempt to capture a complete image of the ...
3
votes
0answers
85 views
Make Mathematica aware of private root certificate authority
I have a private root certificate authority and when I use functions like URLRead[] to connect to a local HTTPS endpoint, I have to add ...
4
votes
2answers
377 views
SSL errors when using URLFetch or URLSave in versions 8 or 9
It seems that with Mathematica 8 and 9 it is not possible anymore to
download files from SSL encrypted urls. In my case (Fedora 30) on Mathematica 8
(it has no ...
2
votes
1answer
592 views
URLSave in Mathematica 10 and 11 on Linux [closed]
It seems that URLSave doesn't work as expected with https links in Mathematica 10.4
on Linux. At least in my case (Fedora 30) for
...
2
votes
1answer
163 views
Convert Plot to Base64 [closed]
My goal is to get a Base64 string directly from a plot (or any Graphics object) in Mathematica such that, when using a Base64 decoder online, such as this one, I ...
5
votes
1answer
168 views
Using WebExecute to automatically download data
With WebExcute, I have been able to get access to a particular url https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx
However, in this url, there is a quite a number of stock code that ...
4
votes
0answers
191 views
Privacy in web-queries [closed]
The question What information is sent to Wolfram Research when loading or working with Mathematica? describes the information sent to Wolfram's servers in routine Mathematica usage (Documentation ...
3
votes
0answers
166 views
How to use Mathematica in Chrome extension?
Now that the wolfram engine is free (for developers), I'd like to know how to use it within a chrome extension. It's easy to make a chrome extension, which is essentially just a JSON file with some ...
0
votes
0answers
78 views
Yahoo groups is going away: how can I download the old messages?
It looks like all the old email messages from my group are stored in web pages with URLs that look like this:
...
10
votes
1answer
221 views
Mathematica 12: Failure with URLRead
Bug introduced in 12.0
The following HTTP request fails to submit the body under version 12, but works in Mathematica 11:
...
12
votes
1answer
362 views
How to work with ElasticSearch in Mathematica?
Steps to install and run ElasticSearch:
install Java (64-bit): https://www.java.com/en/download/manual.jsp
download ElasticSearch (https://www.elastic.co/start) and extract archive in ...
4
votes
2answers
320 views
Can WebExcute click on a button on a webpage that has been created with Javascript?
In an effort to explore the new WebExecute capabilities in Version 12 I have created a simple algorithm to download some data that are stored on an external server and displayed on a webpage. I am ...
1
vote
0answers
85 views
Writing to ElasticSearch [duplicate]
Could someone kindly direct me to suitable documentation, Wolfram or otherwise, to connect Mathematica to an ElasticSearch instance. I don't need anything complicated just sufficient to write a record ...