Questions tagged [data-acquisition]
Questions on the use of Mathematica to capture data from external sources.
113 questions
0
votes
1
answer
91
views
Attempting Data Analysis with FlightData lnformation
I am trying to use data collected from the FlightData[] command to analyze departure delay data for a school project.
I am getting as far as using a command to get the data, taking it into a named ...
- 29.1k
1
vote
1
answer
59
views
How can I use the WikipediaData function to extract certain section information for a list of URL?
I am trying to extract data from a list of URLs for Wikipedia articles. At first, I tried parsing the HTML but it was hard to get the required data as the parsing had some problems and was quite ...
- 619
0
votes
1
answer
99
views
Solve problems from a txt or pdf file
I have tried in vain to recover problems from a flat file and from a pdf and pass them through the reduce option and have it solve the exercises one by one in batches.
Any idea how to achieve it?
The ...
- 29.1k
1
vote
0
answers
65
views
URLRead not working at all
In MMA 13.0 on Mac (Intel silicon) MacSO 12.01
URLRead["https://google.de"]
yields:
...
- 167
10
votes
3
answers
2k
views
Web scraping and multi-threading in Mathematica
I'm working in some web scraping using Mathematica, and today, to speed-up the process, I used bash commands (xargs with parallel options with curl and wget) using ...
- 62.3k
1
vote
1
answer
76
views
Querying ( e.g. ) Google Trends Data with Mathematica front-end
Mathematica has some excellent data query and analysis methods, but can also perform as a front-end to an SQL database.
Having worked as an SQL DBA in the past I would like to investigate further what ...
- 10.3k
3
votes
3
answers
267
views
Recover data from an image of scatter plot
I have this scatter plot and regression line for which I wish I had the original data. I know I can use the coordinates tool in Mathematica to get individual points but that is very tedious and time-...
- 62.3k
1
vote
0
answers
102
views
Import Infinite Scrolling Data
I'm trying to use Import to scrape historical data from Yahoo! finance (update: or any other website that uses infinite scrolling). The issue is that the webpage initially only shows 3 months of data ...
- 269
2
votes
0
answers
101
views
How can I import datasets from TensorFlow datasets into mathematica?
TensorFlow datasets are available at https://www.tensorflow.org/datasets. I wonder if there's a way to directly import any of those datasets directly into Mathematica 12.
I tried
...
- 739
0
votes
0
answers
62
views
scrape App Store web data
I want to scrape the privacy information of an app on the web store page.
For example, the Tiktok app information looks like this:
tiktok on us app store
I found that it's easy to scrape that ...
- 601
1
vote
1
answer
107
views
how to use an unformatted database imported from Excel into Mathematica and draw various charts
In my earlier question (in this link)
the given database is nicely formatted with columns denoting data for variables across countries and years.
But, data sets are not always nicely formatted. Here ...
- 401k
2
votes
0
answers
124
views
BlockchainBlockData request limit exceeded
I'm trying to replicate this Blog entry on Bitcoin queries:
https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/
However, I don't get very far ...
- 601
0
votes
2
answers
739
views
Importing selected data from text files
I have a large number of text files in a folder with different names. These are the output data files from an experiment. Each text file has first 50 lines of text, followed by four columns of ...
- 56.3k
1
vote
0
answers
86
views
Microcontrollerembedcode unable to connect to Arduino Yun
Hi Mathematica enthusiasts,
I am trying to follow this tutorial off Wolfram reference for microcontrollers
Real Time Data Acquisition
I have an Arduino Yun Rev2 hooked up exactly as it is from the ...
- 23
3
votes
2
answers
243
views
How to import a COVID-19 data table from a website into a Mathematica notebook
This link https://www.worldometers.info/coronavirus/ provides daily cases of COVID-19 across all the countries. Is there an easy way to retrieve this table into a Mathematica notebook?
I tried:
...
- 38k
-1
votes
1
answer
156
views
Extract data from picture of experimental instrument
I need to transform many pictures of an analog clock with only on pointer ( the one of hours ) in the corresponding numerical data.
Is it possibile to do it with mathematica? I would like to find ...
- 3,388
2
votes
1
answer
122
views
How to plot data in a web page from a free dataset which updates daily?
I am using the data which is available in the links below to get data about confirmed cases for COVID-19 in Colombia. I am modeling the cumulative cases with a non linear model and all is fine. This ...
- 245k
14
votes
2
answers
767
views
COVID-19 Wolfram Data Repository Retrieval
I'm using this dataset to check out the data on new COVID-19 cases. But there are two issues:
Two days ago it was working great and it had data up to March 13; 1 day lag. Now when I run the cell in ...
- 1
2
votes
2
answers
122
views
How do I perform Bessel function computations for data imported from MS excel?
I am trying to perform Bessel function computations of the form Jn(b*x) where n is the order of the Bessel function. I have three columns representing x, b and n. I wish to import this data into ...
- 109k
1
vote
0
answers
99
views
Finding total Period length of dataset
I've been tasked with trying to do some analysis on some data. I've been given 50000 points of data, which were claimed to have been collected at a discrete period of $1 m s$ Going under this ...
- 3,051
2
votes
3
answers
120
views
Import data from different webpages of the same website
I imported and cleaned the following datasets for england soccer league in order to have home/away rank tables and next round matches:
...
- 401
7
votes
1
answer
459
views
Importing a Wikipedia Table as a Dataset?
I am trying to import this table from Wikipedia into Mathematica as a Dataset. The fields I'm interested are Name, ...
- 62.3k
4
votes
1
answer
129
views
How to find the block number of a transaction in a Wolfram Blockchain?
For example, in block 1004 of the Wolfram Blockchain, I can get the 4th transaction's ID by using the following code:-
...
- 1,018
4
votes
1
answer
256
views
Why is there a lot of noise in the Wolfram Blockchain?
I tried to look into the transaction history of the Wolfram Blockchain. I tried this:-
...
- 1,018
0
votes
0
answers
127
views
Scrape data from Google
my question is general.
I would like to search a term on Google and scrape all the results, including the dates of publishing, if there are.
I know that it is possible to scrape a website - following ...
- 10.3k
1
vote
1
answer
66
views
How to change the allowed formats in Import[...]? [closed]
I want to read data from files with an unusual extension with the Import function. However, the following message appears:
The data is well organized, when I rename the extension to ".dat" ...
- 401k
1
vote
0
answers
82
views
Wikipedia Edits History: how to split edit history and comments
I am trying to import Wikipedia Edit History. The code that I am using is the following:
...
- 11
7
votes
3
answers
667
views
Import data from a current web session?
Consider opening a web session such as (Mathematica 11.3)
session = StartExternalSession["WebDriver-Chrome"];
ffoxdo[cmd_] := ExternalEvaluate[session, cmd];
...
- 9,831
16
votes
3
answers
9k
views
How to request financial data from Yahoo's YQL and Quandl?
How can I download historical data from Yahoo ? Although Mathematica's FinancialData function uses Yahoo it could be useful to have more control on how to retrieve data from Yahoo's YQL.
Edit ...
- 19.1k
19
votes
2
answers
2k
views
Data acquisition from a Measurement Computing USB device
I would like to use Mathematica 9 for to acquire data from a Measurement Computing (MC) USB-201 analog to digital (and other) device for classroom demonstration. I do not have a budget to buy LabView ...
- 21
1
vote
1
answer
332
views
Plot Serial data from Arduino with time date problem
My WM code reads Arduino data from three analog sensors. My code successfully captured the data from the buffer and add the date of capture using the Now[] function....
- 1,438
2
votes
0
answers
119
views
Flight information data
I would like to write a program that automatically checks whether a given flight number is correct and the corresponding departure and arrival times.
Is there any function from the Wolfram Language ...
- 293
0
votes
0
answers
206
views
Extract data (digitise) from graph with various line formats
I am attempting to extract the data points from the graph below:
-axis titles and axis lines are removed for digitising simplicity-
I have tried using a range of graph digtisers (i.e. datathief, ...
- 236k
13
votes
1
answer
927
views
How can I extract data from a diagram?
Below you will see a diagram containing three curves.
How can I extract data points from the three colored curves? I need to use the original data.
- 6,757
6
votes
2
answers
2k
views
Merging Vertices of a Graph and adding up the edges weights
I have a Weighted Graph of g which looks like this.
...
- 236k
41
votes
2
answers
4k
views
How to make a curve selectable from a scaned image and convert it to a list of coordinates
I have a scanned image (binary-ized):
Is there any way to reduce one of these curved lines (full or dotted) to a series of its coordinates (e.g., sampling interval of 0.01 on x-axis)?
I've read ...
- 62.3k
6
votes
1
answer
421
views
webscraping: Wolfram notebook vs browser
In the Netherlands there is a website Funda.nl There you can find all houses for sale or for rent. I used webscraping as a tool to collect data for analyse-purpose. The script I used is like:
...
- 62.3k
6
votes
1
answer
239
views
Export Data from several tabs in a website
Here is the website from the official federation:
http://results.ittf.link/index.php?option=com_fabrik&view=list&listid=36&Itemid=158
I want to export the Data (the table). There are ...
- 62.3k
19
votes
1
answer
2k
views
Release the web camera after using CurrentImage[]
CurrentImage[] will capture an image from the webcam. It is even possible to create a live video in the notebook using
...
0
votes
0
answers
662
views
HTTPRequest and Verbose cURL (curl -vX ...)
Using CouchDB, Windows 10, M11.3
This curl command works fine at the command line,
curl -X PUT http://127.0.0.1:5984/dbName
and this works fine with
...
- 146
6
votes
0
answers
180
views
Connecting to Twitter API Manually in Mathematica to Pull Down All Followers of a Specified User (e.g. More than 5,000)
I have been going through old StackExchange threads to see how to potentially use Mathematica's built-in connection/authentication to the Twitter API to connect to it manually, and access some ...
- 81
0
votes
1
answer
68
views
Fine control of AudioRecord
I use an external sound card (Fireface UCX) for data acquisition, generating stimulus .wav files with Mathematica, using Audacity to simultaneously play the stimulus and record several channels of ...
- 3,436
3
votes
1
answer
819
views
Convert a colormap image into data values according to a legend
I have this image
that I want to turn into data using this gradient
black being 0, white being 1, and all the colors in between accordingly, so I could later use ...
- 24.3k
2
votes
2
answers
344
views
1
vote
1
answer
151
views
Pulling Tabular Data from a Website
I am trying to extract data from this website by simply using
...
- 67.7k
13
votes
1
answer
291
views
Performance-driven approach to using Big Data without killing the hard-drive
Situation
A calculation has to be done in which the amount of data is too much to fit into memory. Fortunately, only a part of the overall data set has to be known at a time. The preprocessing is ...
- 9,568
0
votes
0
answers
60
views
I want to perform mean of a set of data after importing these data from computer in mathematica. But unable. pls help
I have used the following code in mathematica to perform mean to set of data.
...
- 552
4
votes
0
answers
401
views
How can I import from a usb webcam on the raspberry pi?
I'm trying to set up a raspberry pi as a webcam server and I need to import images into a Mathematica notebook on the raspberry pi. I tried using CurrentImage[] ...
CommunityBot
- 1
7
votes
1
answer
779
views
Using sockets to connect to a trading platform
I'm trying to use Binance's API. The following works:
...
- 47.1k
9
votes
2
answers
911
views
What's the fastest way to read a simple numerical list?
I have a large .txt file for which I need to do some data analysis. I've been working on it for a while, and the analysis part has got faster and faster so that most of the evaluation time is now ...
- 291