Questions tagged [data-acquisition]
Questions on the use of Mathematica to capture data from external sources.
113 questions
0
votes
1
answer
91
views
Attempting Data Analysis with FlightData lnformation
I am trying to use data collected from the FlightData[] command to analyze departure delay data for a school project.
I am getting as far as using a command to get the data, taking it into a named ...
- 1
1
vote
1
answer
59
views
How can I use the WikipediaData function to extract certain section information for a list of URL?
I am trying to extract data from a list of URLs for Wikipedia articles. At first, I tried parsing the HTML but it was hard to get the required data as the parsing had some problems and was quite ...
- 33
0
votes
1
answer
99
views
Solve problems from a txt or pdf file
I have tried in vain to recover problems from a flat file and from a pdf and pass them through the reduce option and have it solve the exercises one by one in batches.
Any idea how to achieve it?
The ...
- 2,355
1
vote
0
answers
65
views
URLRead not working at all
In MMA 13.0 on Mac (Intel silicon) MacSO 12.01
URLRead["https://google.de"]
yields:
...
- 167
1
vote
1
answer
76
views
Querying ( e.g. ) Google Trends Data with Mathematica front-end
Mathematica has some excellent data query and analysis methods, but can also perform as a front-end to an SQL database.
Having worked as an SQL DBA in the past I would like to investigate further what ...
- 9,687
3
votes
3
answers
267
views
Recover data from an image of scatter plot
I have this scatter plot and regression line for which I wish I had the original data. I know I can use the coordinates tool in Mathematica to get individual points but that is very tedious and time-...
- 950
1
vote
0
answers
102
views
Import Infinite Scrolling Data
I'm trying to use Import to scrape historical data from Yahoo! finance (update: or any other website that uses infinite scrolling). The issue is that the webpage initially only shows 3 months of data ...
- 269
2
votes
0
answers
101
views
How can I import datasets from TensorFlow datasets into mathematica?
TensorFlow datasets are available at https://www.tensorflow.org/datasets. I wonder if there's a way to directly import any of those datasets directly into Mathematica 12.
I tried
...
- 739
0
votes
0
answers
62
views
scrape App Store web data
I want to scrape the privacy information of an app on the web store page.
For example, the Tiktok app information looks like this:
tiktok on us app store
I found that it's easy to scrape that ...
- 601
1
vote
1
answer
107
views
how to use an unformatted database imported from Excel into Mathematica and draw various charts
In my earlier question (in this link)
the given database is nicely formatted with columns denoting data for variables across countries and years.
But, data sets are not always nicely formatted. Here ...
- 6,409
2
votes
0
answers
124
views
BlockchainBlockData request limit exceeded
I'm trying to replicate this Blog entry on Bitcoin queries:
https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/
However, I don't get very far ...
- 601
0
votes
2
answers
739
views
Importing selected data from text files
I have a large number of text files in a folder with different names. These are the output data files from an experiment. Each text file has first 50 lines of text, followed by four columns of ...
- 1,225
1
vote
0
answers
86
views
Microcontrollerembedcode unable to connect to Arduino Yun
Hi Mathematica enthusiasts,
I am trying to follow this tutorial off Wolfram reference for microcontrollers
Real Time Data Acquisition
I have an Arduino Yun Rev2 hooked up exactly as it is from the ...
- 23
3
votes
2
answers
243
views
How to import a COVID-19 data table from a website into a Mathematica notebook
This link https://www.worldometers.info/coronavirus/ provides daily cases of COVID-19 across all the countries. Is there an easy way to retrieve this table into a Mathematica notebook?
I tried:
...
- 6,409
-1
votes
1
answer
156
views
Extract data from picture of experimental instrument
I need to transform many pictures of an analog clock with only on pointer ( the one of hours ) in the corresponding numerical data.
Is it possibile to do it with mathematica? I would like to find ...
2
votes
1
answer
122
views
How to plot data in a web page from a free dataset which updates daily?
I am using the data which is available in the links below to get data about confirmed cases for COVID-19 in Colombia. I am modeling the cumulative cases with a non linear model and all is fine. This ...
- 500
14
votes
2
answers
767
views
COVID-19 Wolfram Data Repository Retrieval
I'm using this dataset to check out the data on new COVID-19 cases. But there are two issues:
Two days ago it was working great and it had data up to March 13; 1 day lag. Now when I run the cell in ...
- 1,854
2
votes
2
answers
122
views
How do I perform Bessel function computations for data imported from MS excel?
I am trying to perform Bessel function computations of the form Jn(b*x) where n is the order of the Bessel function. I have three columns representing x, b and n. I wish to import this data into ...
- 655
1
vote
0
answers
99
views
Finding total Period length of dataset
I've been tasked with trying to do some analysis on some data. I've been given 50000 points of data, which were claimed to have been collected at a discrete period of $1 m s$ Going under this ...
- 3,051
2
votes
3
answers
120
views
Import data from different webpages of the same website
I imported and cleaned the following datasets for england soccer league in order to have home/away rank tables and next round matches:
...
- 401
7
votes
1
answer
459
views
Importing a Wikipedia Table as a Dataset?
I am trying to import this table from Wikipedia into Mathematica as a Dataset. The fields I'm interested are Name, ...
- 3,175
0
votes
0
answers
127
views
Scrape data from Google
my question is general.
I would like to search a term on Google and scrape all the results, including the dates of publishing, if there are.
I know that it is possible to scrape a website - following ...
- 217
1
vote
1
answer
66
views
How to change the allowed formats in Import[...]? [closed]
I want to read data from files with an unusual extension with the Import function. However, the following message appears:
The data is well organized, when I rename the extension to ".dat" ...
1
vote
0
answers
82
views
Wikipedia Edits History: how to split edit history and comments
I am trying to import Wikipedia Edit History. The code that I am using is the following:
...
- 11
7
votes
3
answers
667
views
Import data from a current web session?
Consider opening a web session such as (Mathematica 11.3)
session = StartExternalSession["WebDriver-Chrome"];
ffoxdo[cmd_] := ExternalEvaluate[session, cmd];
...
- 12.1k
2
votes
0
answers
119
views
Flight information data
I would like to write a program that automatically checks whether a given flight number is correct and the corresponding departure and arrival times.
Is there any function from the Wolfram Language ...
- 293
1
vote
1
answer
332
views
Plot Serial data from Arduino with time date problem
My WM code reads Arduino data from three analog sensors. My code successfully captured the data from the buffer and add the date of capture using the Now[] function....
- 1,438
0
votes
0
answers
206
views
Extract data (digitise) from graph with various line formats
I am attempting to extract the data points from the graph below:
-axis titles and axis lines are removed for digitising simplicity-
I have tried using a range of graph digtisers (i.e. datathief, ...
- 1
13
votes
1
answer
927
views
How can I extract data from a diagram?
Below you will see a diagram containing three curves.
How can I extract data points from the three colored curves? I need to use the original data.
- 131
6
votes
1
answer
421
views
webscraping: Wolfram notebook vs browser
In the Netherlands there is a website Funda.nl There you can find all houses for sale or for rent. I used webscraping as a tool to collect data for analyse-purpose. The script I used is like:
...
- 2,855
6
votes
1
answer
239
views
Export Data from several tabs in a website
Here is the website from the official federation:
http://results.ittf.link/index.php?option=com_fabrik&view=list&listid=36&Itemid=158
I want to export the Data (the table). There are ...
- 1,771
0
votes
0
answers
662
views
HTTPRequest and Verbose cURL (curl -vX ...)
Using CouchDB, Windows 10, M11.3
This curl command works fine at the command line,
curl -X PUT http://127.0.0.1:5984/dbName
and this works fine with
...
- 146
6
votes
0
answers
180
views
Connecting to Twitter API Manually in Mathematica to Pull Down All Followers of a Specified User (e.g. More than 5,000)
I have been going through old StackExchange threads to see how to potentially use Mathematica's built-in connection/authentication to the Twitter API to connect to it manually, and access some ...
- 81
4
votes
1
answer
256
views
Why is there a lot of noise in the Wolfram Blockchain?
I tried to look into the transaction history of the Wolfram Blockchain. I tried this:-
...
- 3,509
4
votes
1
answer
129
views
How to find the block number of a transaction in a Wolfram Blockchain?
For example, in block 1004 of the Wolfram Blockchain, I can get the 4th transaction's ID by using the following code:-
...
- 3,509
3
votes
1
answer
819
views
Convert a colormap image into data values according to a legend
I have this image
that I want to turn into data using this gradient
black being 0, white being 1, and all the colors in between accordingly, so I could later use ...
- 45
2
votes
2
answers
344
views
1
vote
1
answer
151
views
Pulling Tabular Data from a Website
I am trying to extract data from this website by simply using
...
- 175
0
votes
0
answers
60
views
I want to perform mean of a set of data after importing these data from computer in mathematica. But unable. pls help
I have used the following code in mathematica to perform mean to set of data.
...
- 552
0
votes
1
answer
68
views
Fine control of AudioRecord
I use an external sound card (Fireface UCX) for data acquisition, generating stimulus .wav files with Mathematica, using Audacity to simultaneously play the stimulus and record several channels of ...
- 3,436
7
votes
1
answer
779
views
Using sockets to connect to a trading platform
I'm trying to use Binance's API. The following works:
...
- 14.5k
9
votes
2
answers
911
views
What's the fastest way to read a simple numerical list?
I have a large .txt file for which I need to do some data analysis. I've been working on it for a while, and the analysis part has got faster and faster so that most of the evaluation time is now ...
- 291
14
votes
2
answers
620
views
Extracting data from analog screen photograph
I'm using an analog profilometer from the 1970s and need a way to digitize the data and the only way is through a photograph.
I've searched but all I could find was how to extract data from computer ...
- 143
1
vote
1
answer
812
views
Extract data from an image of a density plot?
I have a density plot in image form and would like to extract the data points from it. How would I go about using Mathematica to accomplish this? My ideal output ...
- 1,890
6
votes
1
answer
1k
views
How to plot a list of value from a .dat file?
So I received from a professor a .dat file, which does contain a quite large amount of points. The problem is this: this file is structured as I'm going to show in figure below:
The first value at ...
- 963
0
votes
0
answers
130
views
Problems with Importing a pdf from an url
I have to process a lot of information from different pdf. I already have the code to download all the files, but i'm stuck in the part where i have to open the files with "Import".
For example:
http:...
- 153
1
vote
0
answers
40
views
Monitor and record ethernet port upload speeds
Can I use Mathematica to monitor and record ethernet port upload speeds?
I briefly looked at DeviceStream; not sure if that can be made to work or if that's the ...
- 11
0
votes
1
answer
596
views
How to make a reducer in mathematica (mapreduce)
I want to make a toy mapreduce demonstration in Mathematica using iterator.
Iterator allows to simulate real case when we get input row by row, not nowing in advance how many rows are there in source....
- 619
4
votes
4
answers
1k
views
How to efficiently read blank line separated data file into list of lists
For example, I have a test.dat file with contents as below:
1 2
3 4
5 6
11 22
33 44
55 66
(Note that there could be ...
- 17.4k
3
votes
1
answer
287
views
Parsing JavaScript Script from XML Trees
I've been trying to parse some script written in JavaScript while scraping a website, and am a little confused what functions are needed to get this done in Mathematica.
I imported the data using <...
- 647