Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica
40
votes
3answers
2k views
Recovering data points from an image
Unfortunately, some data can only be obtained in picture form (Japanese publications anyone?). Since this cannot be worked with, it has to be converted to a dataset that can; I was wondering whether ...
18
votes
5answers
2k views
How do you convert a string containing a number in C scientific notation to a Mathematica number?
Suppose I have a string containing the C-representation of a floating point number, for example
s = "1.23e-5"
and I want to convert this to a Mathematica number. ...
16
votes
3answers
609 views
Reading periodic elements from a large file
I have a large binary data file (big endian) with 100+ million "rows" of 11 elements, combination of floats and integers.
This is the format:
...
20
votes
3answers
839 views
How to read data file quickly?
I have a tab separated value file with 10 million rows each of which has three tab separated values. The first value is a string, the second an integer, and the third another string. How to read ...
1
vote
1answer
317 views
How can I speed up image importing? [closed]
The following examples show how slow it is to load a simple GIF. Can anything be done to make it faster?
System #1: Mathematica 8.0.1.0, HP notebook, Win7 64Bit, 2 x 2.4Ghz Intel Core i5, 2GB RAM, w/ ...
25
votes
5answers
2k views
How can I extract data points from a black and white image?
Consider the following graph (source):
Is there any way to extract the data points from this image in a semi-automatic way? I have seen, and tried the methods suggested in this question, but they ...
19
votes
2answers
690 views
Extracting values from nested rules in JSON data
I have been using Mathematica to analyse some data from the StackExchange API. It is conveniently available in JSON form, which Mathematica interprets as replacement rules. However, some of the rules ...
13
votes
3answers
420 views
How to scrape the headlines from New York Times and Wall Street Journal?
How to scrape headlines from New York Times, Wall Street Journal main pages to create datasets similar to this service?
Importing HTML from nyt.com (HTML4) results in a ...
11
votes
1answer
306 views
Release the web camera after using CurrentImage[]
CurrentImage[] will capture an image from the webcam. It is even possible to create a live video in the notebook using
...
6
votes
1answer
1k views
How to extract sample values from an imported WAV file?
Suppose I read a file with
B = Import["something.wav"]
Now how can I obtain a list of sample values?
12
votes
2answers
644 views
How to save all data in all variables so that loading it is fast?
I've got this CSV file I've imported that has tens of millions of lines in it. It takes around 20 minutes to import. I've been working with it for a while and have the processed data spread out in a ...
13
votes
1answer
291 views
Gravatars/identicons with Mathematica
The default avatar/identicon is based on an MD5 hash of your e-mail address. With some e-mail providers (e.g., GMail) the address gmailuser+foo@gmail.com is still ...
12
votes
2answers
346 views
How to develop an Import/Export converter for Compress[]ed data?
Mathematica has a cross platform data exchange format, WDX. Unfortunately importing or exporting large data from/to WDX is very slow. Using MX files is very fast, but they are not compatible across ...
15
votes
1answer
309 views
When importing GIF animation, how to find the correct list of “DisplayDurations”?
When importing GIF animations with variable frame durations, the settings for "DisplayDurations" are not recovered correctly. As an example, I'm using the movie from this earlier question:
Here I'm ...
12
votes
2answers
302 views
How to monitor the progress of Read?
I use
f = OpenRead["largefile.m"]
x = Read[f];
Close[f];
to read in a Mathematica expression from a large file largefile.m ...
9
votes
2answers
894 views
How to get Matlab data imported with the same dimensions?
I have some Matlab image data with the following dimensions (output is from Matlab):
>> size(im)
ans =
86 86 3 45
The data imports just fine ...
3
votes
2answers
803 views
Importing .txt or .csv financial data
I am trying to import some financial data exported from Excel. You can find the relevant file here : Text File or CSV File
This is what the data looks like in Excel :
Using the following code to ...
15
votes
1answer
282 views
Importing Zip files
Mathematica can import zipped files automatically, but when the content of the zip file it's another zipped file, I just get the list of files in the second zip:
...
9
votes
3answers
485 views
Is it possible to import dates and times directly as AbsoluteTime and by pass DateLists?
Date and time calculations are quite slow so I am exploring ways to minimize the number of calculations performed. As an example:
...
4
votes
1answer
528 views
Difficulties with Importing PDFs in Mathematica
I am trying to import a group of PDFs that I have downloaded (it is in the realm of 1,000+). They are a group of doctoral dissertations that I'm trying to automatically extract bibliographic ...
5
votes
1answer
435 views
Combine multiple PDF files into one document
I have multiple single page PDFs I'd like to join together into a single file. (Obviously I mean the file format and not the probability distributions here!)
I'm aware of free online services such as ...
3
votes
1answer
141 views
Exporting lists to files
I am trying to write a program which will generate a list of objects and then exports it in a separate file and clears its memory. As an example I tried the following
...
1
vote
1answer
196 views
How to set path of IMPORT from excel into Mathematica?
I was able to IMPORT from excel data at the start of my project when both the .nb and EXCEL XLS files were sitting in MyDocument (or some other location), but upon moving both files into my Dropbox ...
21
votes
3answers
637 views
How to do http POST in Mathematica?
The Import command lets you do a HTTP GET from a URL, like this
Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg"
This sends an ...
10
votes
1answer
662 views
Web Browsing Automation with Mathematica
I want to build a bot to automate web browsing, this mean something like:
filling forms
press "submit" buttons
find certain text inside pages
and so on...
How can I do this with Mathematica?
The ...
20
votes
1answer
795 views
Programmatically retrieving e-mails from a remote server
Mathematica has a function for sending mail (SendMail) and can import the MBOX format. But what I'd like to do is ...
13
votes
3answers
618 views
Is it possible to use Mathematica to record sound and/or vision?
I think the title says it all. I know about ImageCapture which seems to capture stills.
Can ImageCapture be made to capture a ...
19
votes
3answers
737 views
Importing videos in Mathematica
Mathematica's Import command is purportedly able to import .AVI files. However, I find that many files that I want to import do only result in MMA showing a blank ...
14
votes
1answer
666 views
How to manipulate web pages on Mathematica?
I wanted to input some data on this page through Mathematica:
http://www.fundamentus.com.br/buscaavancada.php
Then I would press the "buscar" button also through mathematica coding and get the ...
8
votes
3answers
860 views
Excel Headers --> Mathematica
I have a large Excel .xls file. Miniature version shown below:
(formatting should appear as two columns...)
...
12
votes
4answers
563 views
Cut and paste data from a spreadsheet
I have a set of data points in two columns in a spreadsheet (OpenOffice Calc):
I would like to get these into Mathematica in this format:
...
12
votes
2answers
649 views
How to import all files of a folder at once?
I know how to import one textfile by calling its name
filestring = Import["myfile.tex", "Text"];
Then "filestring" is a string with the myfile content.
How do ...
8
votes
1answer
261 views
Dealing with large files
I have a *.dat file in which the contents are basically one giant list:
{1,2,3,4} {5,6,7,8} {9,10,11,12}
and so on. I'd like to read in the first 500 members { ...
8
votes
1answer
265 views
How to properly DumpSave & Get Global`s symbols inside packages while not touching Global context?
For efficiency reasons I prefer to use DumpSave instead of Save.
For ease of access I prefer to save symbols in ...
5
votes
1answer
335 views
Exporting data from Excel into Mathematica using clipboard
I know that it's possible to import Excel data into Mathematica using the Import command, as we can see in this post. But some times, I need just to do some fast ...
10
votes
4answers
447 views
Import[ ] drops leading zeros from data when importing a CSV file
I need to load data from the CSV file shown below.
Import[ ] drops the leading zeros in column 3 when it imports the CSV file, but they are security identifiers, and I need them. ReadList[ ] ...
9
votes
2answers
158 views
How to extract a specific element from EXIF photographic metadata?
It's now possible to import photographs into Mathematica and import the EXIF data at the same time:
i = Import["ExampleData/coneflower.jpg", "ImageWithExif"]
...
8
votes
3answers
542 views
Importing HDF5 with compound data
I am looking to Import HDF5 data with compound data. It appears, at least from what I read, that Mathematica 8.0.4 (and previous versions) does not support reading ...
4
votes
1answer
333 views
Efficiently exporting many images as a video
I have many .gif images and I would like to export them as a single video file (I prefer .mov, but any video format is fine). But what if I have several tens of megabytes of .gif images? Is it ...
14
votes
2answers
300 views
Is Compress[] compatible between different Mathematica versions?
Compress[] was introduced in version 6 of Mathematica. It gives a compressed version of expressions represented as a string of ASCII characters. It preserves more ...
9
votes
1answer
341 views
How to import date (YYYY-MM-DD) data from a file?
I have a file with data in form {date, value}:
2010-05-19 17
2010-05-20 20
...
7
votes
1answer
112 views
Organizing similar datasets using Contexts or otherwise
I often import several long time series to analyze something about a city. When I have multiple cities, I use contexts. As a simple example,
...
6
votes
2answers
277 views
How do I display imported images at actual size?
I know that this is an extremely basic question, but I am having a lot of trouble finding the answer in the documentation.
When I import an image (for example, ...
4
votes
1answer
510 views
When do I use Mathematica, when do I use Gephi, and how do I use them together?
Mathematica has great operations for analyzing graphs but when the graphs are a bit bigger, Gephi seems to be more efficient (at least on my computer) at drawing and analyzing, i.e. visualizing the ...
3
votes
2answers
2k views
Importing from Excel
I'm trying to import a matrix from Excel to Mathematica. My code is:
Import["desktop/stproj.xls", "xls", data, 1]
The output is some weird stuff about population ...
2
votes
2answers
150 views
3D scatter plot from a txt file
I have a txt file with space-delimited x, y, z coordinates of several points. The points are organized into groups of unequal size; the groups are delimited by skipping a line, like this:
...
10
votes
3answers
1k views
Plot data from txt file?
I have data in file.txt which can be expressed by 20000x11 matrix. I do not want to import all of these data. How can I plot graph of the column 1 & column 4?
7
votes
1answer
645 views
Import DCD (CHARMM/NAMD binary trajectory) file
How can one import the coordinates stored in a binary CHARMM/NAMD DCD file?
This is the structure of the file:
...
6
votes
3answers
318 views
Efficient image Import
I have several sets of 20 images I need to import.
Each time they have different categories.
"Name"<>"Cat"<>"No"
Electric B 1 for example.
I need to create lists that contain the images ...
4
votes
1answer
153 views
How can I colorize an imported model?
How can I colorize an imported model from for example 3ds max?
I'm trying to find the way to colorize a 3D plot from an imported model with something like this:
...
