Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

76
votes
3answers
8k views

Recovering data points from an image

Unfortunately, some data can only be obtained in picture form (Japanese publications anyone?). Since this cannot be worked with, it has to be converted to a dataset that can; I was wondering whether ...
44
votes
5answers
6k views

How can I extract data points from a black and white image?

Consider the following graph (source): Is there any way to extract the data points from this image in a semi-automatic way? I have seen, and tried the methods suggested in this question, but they ...
40
votes
8answers
8k views

How do you convert a string containing a number in C scientific notation to a Mathematica number?

Suppose I have a string containing the C-representation of a floating point number, for example s = "1.23e-5" and I want to convert this to a Mathematica number. ...
37
votes
3answers
3k views

How to do http POST in Mathematica?

The Import command lets you do a HTTP GET from a URL, like this Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg" This sends an ...
37
votes
4answers
3k views

Importing videos in Mathematica

Mathematica's Import command is purportedly able to import .AVI files. However, I find that many files that I want to import do only result in MMA showing a blank ...
37
votes
1answer
2k views

Web Browsing Automation with Mathematica

I want to build a bot to automate web browsing, this mean something like: filling forms press "submit" buttons find certain text inside pages and so on... How can I do this with Mathematica? The ...
29
votes
4answers
2k views

How can I import a huge CSV file quickly?

I have two CSV files, each one is around 1 GB of data. When I use Import["file.csv"], it takes a very, very, long time to import the data. So how can I accelerate ...
29
votes
5answers
4k views

Importing a large Excel file

I'm trying to import a simple Excel file containing 94,000 rows and 52 columns of various types, mostly Reals. It's a 38-MB file. I get this super helpful error ...
28
votes
2answers
3k views

How to build a bvh (a motion capture file format) player in Mathematica?

How to build a bvh player that can be used to visualize human motions in Mathematica? Any idea or sample code would be appreciated. ...
28
votes
3answers
3k views

How to read data file quickly?

I have a tab separated value file with 10 million rows each of which has three tab separated values. The first value is a string, the second an integer, and the third another string. How to read ...
28
votes
1answer
4k views

Programmatically retrieving e-mails from a remote server

Mathematica has a function for sending mail (SendMail) and can import the MBOX format. But what I'd like to do is ...
27
votes
2answers
3k views

How to save all data in all variables so that loading it is fast?

I've got this CSV file I've imported that has tens of millions of lines in it. It takes around 20 minutes to import. I've been working with it for a while and have the processed data spread out in a ...
27
votes
3answers
2k views

How to manipulate web pages on Mathematica?

I wanted to input some data on this page through Mathematica. Then I would press the "buscar" button also through Mathematica coding and get the results on Mathematica. The most similar post I've ...
26
votes
3answers
3k views

Extracting values from nested rules in JSON data

I have been using Mathematica to analyse some data from the StackExchange API. It is conveniently available in JSON form, which Mathematica interprets as replacement rules. However, some of the rules ...
25
votes
3answers
2k views

Finding a function that fits the shape of an image

How can I find a function that fits the given shape of an image? Or instead extract a point set from the shape and then find an interpolating function fits the shape. An example image is attached.
25
votes
1answer
330 views

Getting Mathematica.SE reputation data

I'm asking the following on behalf of Mr.Wizard. On StackOverflow I used the old v1.0 SO API and Mathematica to obtain user reputation data. Now we are no longer a tag on StackOverflow and the v2.1 ...
24
votes
6answers
5k views

Cut and paste data from a spreadsheet

I have a set of data points in two columns in a spreadsheet (OpenOffice Calc): I would like to get these into Mathematica in this format: ...
24
votes
2answers
864 views

Import an RDATA, SPSS, SAS, or STATA file into Mathematica

I would like to import the World Value Survey data file into Mathematica, but it's only given in spss, sas or stata formats, which are statistical analysis softwares format I think. I can't get to ...
23
votes
2answers
6k views

How to import all files of a folder at once?

I know how to import one textfile by calling its name filestring = Import["myfile.tex", "Text"]; Then "filestring" is a string with the myfile content. How do ...
23
votes
3answers
2k views

Reading periodic elements from a large file

I have a large binary data file (big endian) with 100+ million "rows" of 11 elements, combination of floats and integers. This is the format: ...
21
votes
4answers
2k views

How to efficiently import videos

I need to do frame-by-frame analysis of videos. I can import the video files I need, but each file is too large to import at once. I can use the ...
21
votes
2answers
484 views

Is Compress[] compatible between different Mathematica versions?

Compress[] was introduced in version 6 of Mathematica. It gives a compressed version of expressions represented as a string of ASCII characters. It preserves more ...
20
votes
4answers
2k views

Is it possible to use Mathematica to record sound and/or vision?

I think the title says it all. I know about ImageCapture which seems to capture stills. Can ImageCapture be made to capture a ...
20
votes
2answers
671 views

How to develop an Import/Export converter for Compress[]ed data?

Mathematica has a cross platform data exchange format, WDX. Unfortunately importing or exporting large data from/to WDX is very slow. Using MX files is very fast, but they are not compatible across ...
20
votes
2answers
897 views

Why does ImageData need four times more memory?

Background We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
19
votes
1answer
818 views

When importing GIF animation, how to find the correct list of “DisplayDurations”?

When importing GIF animations with variable frame durations, the settings for "DisplayDurations" are not recovered correctly. As an example, I'm using the movie from this earlier question: Here I'm ...
19
votes
1answer
2k views

How to programmatically extract mathematical formulas from Word with Mathematica 9?

I have a bunch of(1000+) Microsoft Word document in .Docx format. How can I programmatically extract the mathematical formulas from MS Word using Mathematica 9? This is what example looks like (or ...
19
votes
1answer
556 views

Gravatars/identicons with Mathematica

The default avatar/identicon is based on an MD5 hash of your e-mail address. With some e-mail providers (e.g., GMail) the address gmailuser+foo@gmail.com is still ...
18
votes
1answer
1k views

Slow import of multigigabyte TIF image stacks

Update 6: After contacting Wolfram Technical Support, I was advised that the frame-to-frame import timing variability can be eliminated by reading in the frame stack from frame to frame with the ...
18
votes
0answers
360 views

Getting EPS imports to respect the original fill rule

One major problem in my attempt to answer to the CountryData flags question is that the way Mathematica handles filled curves differs from the way SVG and EPS style editors do. The latter two have the ...
17
votes
1answer
1k views

Release the web camera after using CurrentImage[]

CurrentImage[] will capture an image from the webcam. It is even possible to create a live video in the notebook using ...
16
votes
1answer
511 views

Importing Zip files

Mathematica can import zipped files automatically, but when the content of the zip file it's another zipped file, I just get the list of files in the second zip: ...
16
votes
2answers
1k views

Dealing with large files

I have a *.dat file in which the contents are basically one giant list: {1,2,3,4} {5,6,7,8} {9,10,11,12} and so on. I'd like to read in the first 500 members { ...
16
votes
2answers
221 views

HDF5 importing large arrays

I have some computations that output a 300,000,000 element array of long unsigned ints, so about 2.4GB, in an h5 file. Mathematica handles files south of 2GB fine, albeit taking very long to import ...
15
votes
3answers
11k views

Plot data from txt file?

I have data in file.txt which can be expressed by 20000x11 matrix. I do not want to import all of these data. How can I plot graph of the column 1 & column 4?
15
votes
1answer
341 views

Why does ImportString[“1c”, “Table”] eat the letter c?

I have some whitespace-separated matrix data that I read with Import[..., "Table"]. The data contained mixed strings and numbers (the strings are for row and ...
15
votes
2answers
809 views

Get both output and return code of external program

Run["program"] will run an external program and return the return code. Import["!program", "String"] will run the program and ...
15
votes
3answers
652 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
14
votes
3answers
1k views

How to scrape the headlines from New York Times and Wall Street Journal?

How to scrape headlines from New York Times, Wall Street Journal main pages to create datasets similar to this service? Importing HTML from nyt.com (HTML4) results in a ...
14
votes
2answers
404 views

How to load thumbnail, not image?

Is there any way to import only the thumbnail of an image instead of the full image? I frequently work with lots of images and it would be nice if the contents of a directory could be quickly ...
14
votes
4answers
2k views

Importing HDF5 with compound data

I am looking to Import HDF5 data with compound data. It appears, at least from what I read, that Mathematica 8.0.4 (and previous versions) does not support reading ...
14
votes
2answers
642 views

How to monitor the progress of Read?

I use f = OpenRead["largefile.m"] x = Read[f]; Close[f]; to read in a Mathematica expression from a large file largefile.m ...
14
votes
1answer
858 views

Data acquisition from a Measurement Computing USB device

I would like to use Mathematica 9 for to acquire data from a Measurement Computing (MC) USB-201 analog to digital (and other) device for classroom demonstration. I do not have a budget to buy LabView ...
13
votes
4answers
407 views

Unencoding encoded URLs in a function

Is there a built-in function for decoding encoded URLs in Mathematica? I would like to convert an encoded URL, for example ...
13
votes
3answers
2k views

How can I transfer mathematical expressions between Mathematica and Maple?

Is there a simple way to copy mathematical expressions between Mathematica and Maple (or at least in one direction)? I mean only expressions built from numbers and predefined mathematical functions, ...
13
votes
4answers
1k views

Import[ ] drops leading zeros from data when importing a CSV file

I need to load data from the CSV file shown below. Import[ ] drops the leading zeros in column 3 when it imports the CSV file, but they are security identifiers, and I need them. ReadList[ ] ...
13
votes
1answer
399 views

How can I read compressed .Z file automatically by Mathematica?

When I read .Z file by something like jpldatstring = Import["ftp://cddis.gsfc.nasa.gov/products/jpl/2003/jplg0040.04i.Z"] Unfortunately, it turned out that ...
13
votes
3answers
420 views

ToExpression fails with long nested string expressions

Consider the following conversion from String to Mathematica Expression: ...
13
votes
1answer
423 views

Any solution to transfer data between Python and Mathematica

I think Mathematica handles .mat file not very well, python is pretty good at this kind of jobs. In this site, there are plenty of question related to calling Mathematica from Python, or calling ...
13
votes
2answers
629 views

Problem importing URL with Greek characters

I'm considering buying a car. So I thought why not make a web-crawler in Mathematica to pile-up car data? Brilliant idea. Then I found this Greek website, gocar.gr, which just so happens to have all ...