Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

68
votes
3answers
6k views

Recovering data points from an image

Unfortunately, some data can only be obtained in picture form (Japanese publications anyone?). Since this cannot be worked with, it has to be converted to a dataset that can; I was wondering whether ...
39
votes
8answers
6k views

How do you convert a string containing a number in C scientific notation to a Mathematica number?

Suppose I have a string containing the C-representation of a floating point number, for example s = "1.23e-5" and I want to convert this to a Mathematica number. ...
39
votes
5answers
5k views

How can I extract data points from a black and white image?

Consider the following graph (source): Is there any way to extract the data points from this image in a semi-automatic way? I have seen, and tried the methods suggested in this question, but they ...
36
votes
4answers
3k views

Importing videos in Mathematica

Mathematica's Import command is purportedly able to import .AVI files. However, I find that many files that I want to import do only result in MMA showing a blank ...
34
votes
3answers
2k views

How to do http POST in Mathematica?

The Import command lets you do a HTTP GET from a URL, like this Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg" This sends an ...
32
votes
1answer
2k views

Web Browsing Automation with Mathematica

I want to build a bot to automate web browsing, this mean something like: filling forms press "submit" buttons find certain text inside pages and so on... How can I do this with Mathematica? The ...
26
votes
3answers
3k views

How to read data file quickly?

I have a tab separated value file with 10 million rows each of which has three tab separated values. The first value is a string, the second an integer, and the third another string. How to read ...
25
votes
3answers
2k views

Extracting values from nested rules in JSON data

I have been using Mathematica to analyse some data from the StackExchange API. It is conveniently available in JSON form, which Mathematica interprets as replacement rules. However, some of the rules ...
24
votes
1answer
3k views

Programmatically retrieving e-mails from a remote server

Mathematica has a function for sending mail (SendMail) and can import the MBOX format. But what I'd like to do is ...
23
votes
3answers
2k views

Reading periodic elements from a large file

I have a large binary data file (big endian) with 100+ million "rows" of 11 elements, combination of floats and integers. This is the format: ...
23
votes
2answers
2k views

How to manipulate web pages on Mathematica?

I wanted to input some data on this page through Mathematica: http://www.fundamentus.com.br/buscaavancada.php Then I would press the "buscar" button also through mathematica coding and get the ...
22
votes
2answers
3k views

How to save all data in all variables so that loading it is fast?

I've got this CSV file I've imported that has tens of millions of lines in it. It takes around 20 minutes to import. I've been working with it for a while and have the processed data spread out in a ...
21
votes
2answers
2k views

How to build a bvh (a motion capture file format) player in Mathematica?

How to build a bvh player that can be used to visualize human motions in Mathematica? Any idea or sample code would be appreciated. ...
21
votes
3answers
1k views

Finding a function that fits the shape of an image

How can I find a function that fits the given shape of an image? Or instead extract a point set from the shape and then find an interpolating function fits the shape. An example image is attached.
21
votes
6answers
4k views

Cut and paste data from a spreadsheet

I have a set of data points in two columns in a spreadsheet (OpenOffice Calc): I would like to get these into Mathematica in this format: ...
21
votes
4answers
3k views

Importing a large Excel file

I'm trying to import a simple Excel file containing 94,000 rows and 52 columns of various types, mostly Reals. It's a 38-MB file. I get this super helpful error ...
21
votes
2answers
4k views

How to import all files of a folder at once?

I know how to import one textfile by calling its name filestring = Import["myfile.tex", "Text"]; Then "filestring" is a string with the myfile content. How do ...
21
votes
4answers
1k views

How to efficiently import videos

I need to do frame-by-frame analysis of videos. I can import the video files I need, but each file is too large to import at once. I can use the ...
21
votes
2answers
680 views

Import an RDATA, SPSS, SAS, or STATA file into Mathematica

I would like to import the World Value Survey data file into Mathematica, but it's only given in spss, sas or stata formats, which are statistical analysis softwares format I think. I can't get to ...
20
votes
4answers
2k views

Is it possible to use Mathematica to record sound and/or vision?

I think the title says it all. I know about ImageCapture which seems to capture stills. Can ImageCapture be made to capture a ...
20
votes
1answer
264 views

Getting Mathematica.SE reputation data

I'm asking the following on behalf of Mr.Wizard. On StackOverflow I used the old v1.0 SO API and Mathematica to obtain user reputation data. Now we are no longer a tag on StackOverflow and the v2.1 ...
20
votes
2answers
441 views

Is Compress[] compatible between different Mathematica versions?

Compress[] was introduced in version 6 of Mathematica. It gives a compressed version of expressions represented as a string of ASCII characters. It preserves more ...
19
votes
2answers
627 views

How to develop an Import/Export converter for Compress[]ed data?

Mathematica has a cross platform data exchange format, WDX. Unfortunately importing or exporting large data from/to WDX is very slow. Using MX files is very fast, but they are not compatible across ...
19
votes
2answers
800 views

Why does ImageData need four times more memory?

Background We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
18
votes
1answer
679 views

When importing GIF animation, how to find the correct list of “DisplayDurations”?

When importing GIF animations with variable frame durations, the settings for "DisplayDurations" are not recovered correctly. As an example, I'm using the movie from this earlier question: Here I'm ...
17
votes
1answer
486 views

Gravatars/identicons with Mathematica

The default avatar/identicon is based on an MD5 hash of your e-mail address. With some e-mail providers (e.g., GMail) the address gmailuser+foo@gmail.com is still ...
16
votes
1answer
1k views

How to programmatically extract mathematical formulas from Word with Mathematica 9?

I have a bunch of(1000+) Microsoft Word document in .Docx format. How can I programmatically extract the mathematical formulas from MS Word using Mathematica 9? This is what example looks like (or ...
16
votes
1answer
983 views

Release the web camera after using CurrentImage[]

CurrentImage[] will capture an image from the webcam. It is even possible to create a live video in the notebook using ...
16
votes
2answers
1k views

Dealing with large files

I have a *.dat file in which the contents are basically one giant list: {1,2,3,4} {5,6,7,8} {9,10,11,12} and so on. I'd like to read in the first 500 members { ...
16
votes
1answer
990 views

Slow import of multigigabyte TIF image stacks

Update 6: After contacting Wolfram Technical Support, I was advised that the frame-to-frame import timing variability can be eliminated by reading in the frame stack from frame to frame with the ...
15
votes
1answer
323 views

Why does ImportString[“1c”, “Table”] eat the letter c?

I have some whitespace-separated matrix data that I read with Import[..., "Table"]. The data contained mixed strings and numbers (the strings are for row and ...
15
votes
1answer
445 views

Importing Zip files

Mathematica can import zipped files automatically, but when the content of the zip file it's another zipped file, I just get the list of files in the second zip: ...
15
votes
2answers
691 views

Get both output and return code of external program

Run["program"] will run an external program and return the return code. Import["!program", "String"] will run the program and ...
15
votes
2answers
158 views

HDF5 importing large arrays

I have some computations that output a 300,000,000 element array of long unsigned ints, so about 2.4GB, in an h5 file. Mathematica handles files south of 2GB fine, albeit taking very long to import ...
14
votes
3answers
8k views

Plot data from txt file?

I have data in file.txt which can be expressed by 20000x11 matrix. I do not want to import all of these data. How can I plot graph of the column 1 & column 4?
14
votes
2answers
380 views

How to load thumbnail, not image?

Is there any way to import only the thumbnail of an image instead of the full image? I frequently work with lots of images and it would be nice if the contents of a directory could be quickly ...
14
votes
2answers
572 views

How to monitor the progress of Read?

I use f = OpenRead["largefile.m"] x = Read[f]; Close[f]; to read in a Mathematica expression from a large file largefile.m ...
14
votes
3answers
569 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
13
votes
3answers
1k views

How to scrape the headlines from New York Times and Wall Street Journal?

How to scrape headlines from New York Times, Wall Street Journal main pages to create datasets similar to this service? Importing HTML from nyt.com (HTML4) results in a ...
13
votes
4answers
1k views

Importing HDF5 with compound data

I am looking to Import HDF5 data with compound data. It appears, at least from what I read, that Mathematica 8.0.4 (and previous versions) does not support reading ...
13
votes
2answers
536 views

Problem importing URL with Greek characters

I'm considering buying a car. So I thought why not make a web-crawler in Mathematica to pile-up car data? Brilliant idea. Then I found this Greek website, gocar.gr, which just so happens to have all ...
13
votes
1answer
611 views

Data acquisition from a Measurement Computing USB device

I would like to use Mathematica 9 for to acquire data from a Measurement Computing (MC) USB-201 analog to digital (and other) device for classroom demonstration. I do not have a budget to buy LabView ...
12
votes
4answers
370 views

Unencoding encoded URLs in a function

Is there a built-in function for decoding encoded URLs in Mathematica? I would like to convert an encoded URL, for example ...
12
votes
3answers
1k views

How can I transfer mathematical expressions between Mathematica and Maple?

Is there a simple way to copy mathematical expressions between Mathematica and Maple (or at least in one direction)? I mean only expressions built from numbers and predefined mathematical functions, ...
12
votes
6answers
2k views

How do you skip missing data values in a data file?

In scanning through a simple imported file of weather station data, some dates have missing data. What is a test to use to check whether a datum is valid? I tried ...
12
votes
3answers
4k views

Excel Headers --> Mathematica

I have a large Excel .xls file. Miniature version shown below: (formatting should appear as two columns...) ...
12
votes
2answers
621 views

Parsing a CSV file using Interpreter

The new Interpreter functionality in version 10 looks like it has the potential to make parsing custom data formats very easy. I'm trying to create a CSV parser. ...
12
votes
4answers
1k views

Import[ ] drops leading zeros from data when importing a CSV file

I need to load data from the CSV file shown below. Import[ ] drops the leading zeros in column 3 when it imports the CSV file, but they are security identifiers, and I need them. ReadList[ ] ...
12
votes
3answers
333 views

ToExpression fails with long nested string expressions

Consider the following conversion from String to Mathematica Expression: ...
12
votes
1answer
207 views

Any solution to transfer data between Python and Mathematica

I think Mathematica handles .mat file not very well, python is pretty good at this kind of jobs. In this site, there are plenty of question related to calling Mathematica from Python, or calling ...