Questions tagged [data-acquisition]

Questions on the use of Mathematica to capture data from external sources.

Filter by
Sorted by
Tagged with
1 vote
0 answers
49 views

URLRead not working at all

In MMA 13.0 on Mac (Intel silicon) MacSO 12.01 URLRead["https://google.de"] yields: ...
user avatar
  • 165
0 votes
1 answer
57 views

Querying ( e.g. ) Google Trends Data with Mathematica front-end

Mathematica has some excellent data query and analysis methods, but can also perform as a front-end to an SQL database. Having worked as an SQL DBA in the past I would like to investigate further what ...
user avatar
2 votes
3 answers
142 views

Recover data from an image of scatter plot

I have this scatter plot and regression line for which I wish I had the original data. I know I can use the coordinates tool in Mathematica to get individual points but that is very tedious and time-...
user avatar
  • 579
1 vote
0 answers
54 views

Import Infinite Scrolling Data

I'm trying to use Import to scrape historical data from Yahoo! finance (update: or any other website that uses infinite scrolling). The issue is that the webpage initially only shows 3 months of data ...
user avatar
  • 237
2 votes
0 answers
54 views

How can I import datasets from TensorFlow datasets into mathematica?

TensorFlow datasets are available at https://www.tensorflow.org/datasets. I wonder if there's a way to directly import any of those datasets directly into Mathematica 12. I tried ...
user avatar
  • 483
0 votes
0 answers
44 views

scrape App Store web data

I want to scrape the privacy information of an app on the web store page. For example, the Tiktok app information looks like this: tiktok on us app store I found that it's easy to scrape that ...
user avatar
  • 591
1 vote
1 answer
98 views

how to use an unformatted database imported from Excel into Mathematica and draw various charts

In my earlier question (in this link) the given database is nicely formatted with columns denoting data for variables across countries and years. But, data sets are not always nicely formatted. Here ...
user avatar
  • 4,391
2 votes
0 answers
103 views

BlockchainBlockData request limit exceeded

I'm trying to replicate this Blog entry on Bitcoin queries: https://blog.wolfram.com/2021/05/25/sleuthing-darkside-crypto-ransom-payments-with-the-wolfram-language/ However, I don't get very far ...
user avatar
  • 591
0 votes
2 answers
191 views

Importing selected data from text files

I have a large number of text files in a folder with different names. These are the output data files from an experiment. Each text file has first 50 lines of text, followed by four columns of ...
user avatar
  • 973
1 vote
0 answers
52 views

Microcontrollerembedcode unable to connect to Arduino Yun

Hi Mathematica enthusiasts, I am trying to follow this tutorial off Wolfram reference for microcontrollers Real Time Data Acquisition I have an Arduino Yun Rev2 hooked up exactly as it is from the ...
user avatar
4 votes
2 answers
221 views

How to import a COVID-19 data table from a website into a Mathematica notebook

This link https://www.worldometers.info/coronavirus/ provides daily cases of COVID-19 across all the countries. Is there an easy way to retrieve this table into a Mathematica notebook? I tried: ...
user avatar
  • 4,391
-1 votes
1 answer
116 views

Extract data from picture of experimental instrument

I need to transform many pictures of an analog clock with only on pointer ( the one of hours ) in the corresponding numerical data. Is it possibile to do it with mathematica? I would like to find ...
user avatar
2 votes
1 answer
103 views

How to plot data in a web page from a free dataset which updates daily?

I am using the data which is available in the links below to get data about confirmed cases for COVID-19 in Colombia. I am modeling the cumulative cases with a non linear model and all is fine. This ...
user avatar
  • 480
14 votes
2 answers
741 views

COVID-19 Wolfram Data Repository Retrieval

I'm using this dataset to check out the data on new COVID-19 cases. But there are two issues: Two days ago it was working great and it had data up to March 13; 1 day lag. Now when I run the cell in ...
user avatar
  • 1,448
2 votes
2 answers
113 views

How do I perform Bessel function computations for data imported from MS excel?

I am trying to perform Bessel function computations of the form Jn(b*x) where n is the order of the Bessel function. I have three columns representing x, b and n. I wish to import this data into ...
user avatar
  • 595
1 vote
0 answers
81 views

Finding total Period length of dataset

I've been tasked with trying to do some analysis on some data. I've been given 50000 points of data, which were claimed to have been collected at a discrete period of $1 m s$ Going under this ...
user avatar
2 votes
3 answers
108 views

Import data from different webpages of the same website

I imported and cleaned the following datasets for england soccer league in order to have home/away rank tables and next round matches: ...
user avatar
  • 401
7 votes
1 answer
369 views

Importing a Wikipedia Table as a Dataset?

I am trying to import this table from Wikipedia into Mathematica as a Dataset. The fields I'm interested are Name, ...
user avatar
  • 2,975
0 votes
0 answers
107 views

Scrape data from Google

my question is general. I would like to search a term on Google and scrape all the results, including the dates of publishing, if there are. I know that it is possible to scrape a website - following ...
user avatar
1 vote
1 answer
64 views

How to change the allowed formats in Import[...]? [closed]

I want to read data from files with an unusual extension with the Import function. However, the following message appears: The data is well organized, when I rename the extension to ".dat" ...
user avatar
1 vote
0 answers
74 views

Wikipedia Edits History: how to split edit history and comments

I am trying to import Wikipedia Edit History. The code that I am using is the following: ...
user avatar
  • 11
6 votes
3 answers
562 views

Import data from a current web session?

Consider opening a web session such as (Mathematica 11.3) session = StartExternalSession["WebDriver-Chrome"]; ffoxdo[cmd_] := ExternalEvaluate[session, cmd]; ...
user avatar
  • 11.5k
2 votes
0 answers
88 views

Flight information data

I would like to write a program that automatically checks whether a given flight number is correct and the corresponding departure and arrival times. Is there any function from the Wolfram Language ...
user avatar
  • 283
1 vote
1 answer
231 views

Plot Serial data from Arduino with time date problem

My WM code reads Arduino data from three analog sensors. My code successfully captured the data from the buffer and add the date of capture using the Now[] function....
user avatar
0 votes
0 answers
134 views

Extract data (digitise) from graph with various line formats

I am attempting to extract the data points from the graph below: -axis titles and axis lines are removed for digitising simplicity- I have tried using a range of graph digtisers (i.e. datathief, ...
user avatar
  • 1
11 votes
1 answer
778 views

How can I extract data from a diagram?

Below you will see a diagram containing three curves. How can I extract data points from the three colored curves? I need to use the original data.
user avatar
5 votes
1 answer
341 views

webscraping: Wolfram notebook vs browser

In the Netherlands there is a website Funda.nl There you can find all houses for sale or for rent. I used webscraping as a tool to collect data for analyse-purpose. The script I used is like: ...
user avatar
6 votes
1 answer
192 views

Export Data from several tabs in a website

Here is the website from the official federation: http://results.ittf.link/index.php?option=com_fabrik&view=list&listid=36&Itemid=158 I want to export the Data (the table). There are ...
user avatar
  • 1,609
0 votes
0 answers
554 views

HTTPRequest and Verbose cURL (curl -vX ...)

Using CouchDB, Windows 10, M11.3 This curl command works fine at the command line, curl -X PUT http://127.0.0.1:5984/dbName and this works fine with ...
user avatar
  • 146
5 votes
0 answers
166 views

Connecting to Twitter API Manually in Mathematica to Pull Down All Followers of a Specified User (e.g. More than 5,000)

I have been going through old StackExchange threads to see how to potentially use Mathematica's built-in connection/authentication to the Twitter API to connect to it manually, and access some ...
user avatar
4 votes
1 answer
229 views

Why is there a lot of noise in the Wolfram Blockchain?

I tried to look into the transaction history of the Wolfram Blockchain. I tried this:- ...
user avatar
  • 3,389
4 votes
1 answer
102 views

How to find the block number of a transaction in a Wolfram Blockchain?

For example, in block 1004 of the Wolfram Blockchain, I can get the 4th transaction's ID by using the following code:- ...
user avatar
  • 3,389
3 votes
1 answer
326 views

Convert a colormap image into data values according to a legend

I have this image that I want to turn into data using this gradient black being 0, white being 1, and all the colors in between accordingly, so I could later use ...
user avatar
2 votes
2 answers
262 views

how to drop certain data points from a list

...
user avatar
  • 157
1 vote
1 answer
129 views

Pulling Tabular Data from a Website

I am trying to extract data from this website by simply using ...
user avatar
  • 175
0 votes
0 answers
59 views

I want to perform mean of a set of data after importing these data from computer in mathematica. But unable. pls help

I have used the following code in mathematica to perform mean to set of data. ...
user avatar
  • 309
0 votes
1 answer
61 views

Fine control of AudioRecord

I use an external sound card (Fireface UCX) for data acquisition, generating stimulus .wav files with Mathematica, using Audacity to simultaneously play the stimulus and record several channels of ...
user avatar
  • 3,236
7 votes
1 answer
674 views

Using sockets to connect to a trading platform

I'm trying to use Binance's API. The following works: ...
user avatar
  • 13.6k
9 votes
2 answers
621 views

What's the fastest way to read a simple numerical list?

I have a large .txt file for which I need to do some data analysis. I've been working on it for a while, and the analysis part has got faster and faster so that most of the evaluation time is now ...
user avatar
  • 291
13 votes
2 answers
558 views

Extracting data from analog screen photograph

I'm using an analog profilometer from the 1970s and need a way to digitize the data and the only way is through a photograph. I've searched but all I could find was how to extract data from computer ...
user avatar
  • 133
1 vote
1 answer
583 views

Extract data from an image of a density plot?

I have a density plot in image form and would like to extract the data points from it. How would I go about using Mathematica to accomplish this? My ideal output ...
user avatar
  • 1,800
6 votes
1 answer
1k views

How to plot a list of value from a .dat file?

So I received from a professor a .dat file, which does contain a quite large amount of points. The problem is this: this file is structured as I'm going to show in figure below: The first value at ...
user avatar
  • 713
0 votes
0 answers
110 views

Problems with Importing a pdf from an url

I have to process a lot of information from different pdf. I already have the code to download all the files, but i'm stuck in the part where i have to open the files with "Import". For example: http:...
user avatar
  • 143
1 vote
0 answers
34 views

Monitor and record ethernet port upload speeds

Can I use Mathematica to monitor and record ethernet port upload speeds? I briefly looked at DeviceStream; not sure if that can be made to work or if that's the ...
user avatar
  • 11
0 votes
1 answer
454 views

How to make a reducer in mathematica (mapreduce)

I want to make a toy mapreduce demonstration in Mathematica using iterator. Iterator allows to simulate real case when we get input row by row, not nowing in advance how many rows are there in source....
user avatar
  • 609
4 votes
4 answers
992 views

How to efficiently read blank line separated data file into list of lists

For example, I have a test.dat file with contents as below: 1 2 3 4 5 6 11 22 33 44 55 66 (Note that there could be ...
user avatar
  • 15.6k
3 votes
1 answer
262 views

Parsing JavaScript Script from XML Trees

I've been trying to parse some script written in JavaScript while scraping a website, and am a little confused what functions are needed to get this done in Mathematica. I imported the data using <...
user avatar
2 votes
2 answers
451 views

Smooth curves in a raster image

I have a *.bmp image, e.g. this one: I need to smooth boundaries of the thick curves and save it in a vector graphics form. Is there a simple way how to do that in Mathematica 10? I know it is ...
user avatar
4 votes
1 answer
1k views

How to make Import[...., "Table"] not convert strings to numbers and not convert numbers to strings?

Bug introduced in 8 or earlier and persisting through 11.2.0 I have to to use ...
user avatar
  • 108k
12 votes
1 answer
248 views

Performance-driven approach to using Big Data without killing the hard-drive

Situation A calculation has to be done in which the amount of data is too much to fit into memory. Fortunately, only a part of the overall data set has to be known at a time. The preprocessing is ...
user avatar
  • 822