Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

2
votes
0answers
451 views

Can not Import PDF

I try to import PDF and extract text from it Import["D:\\Vegetarian MS.pdf", {"PDF", "Plaintext"}] but with some files I have a problem like this ...
9
votes
2answers
663 views

How can I animate a set of points imported from file?

I have an external program which generates a set of data - X points at the time t1, X points at the time t2 and so on. These points are stored in CSV file and I can change the format if necessary. ...
6
votes
2answers
273 views

How to process binary-encoded data without writing to file and importing?

I am consuming a webservice through NETLink that returns data directly in the response as a gzipped csv file. I can successfully process the data like this: ...
4
votes
1answer
2k views

Importing CSV financial data in Mathematica

I am trying to import a CSV file in Mathematica 8 with the aim of charting it with the TradingChart command. This is what I have done : ...
4
votes
1answer
608 views

Efficiently exporting many images as a video

I have many .gif images and I would like to export them as a single video file (I prefer .mov, but any video format is fine). But what if I have several tens of megabytes of .gif images? Is it ...
12
votes
2answers
467 views

How to monitor the progress of Read?

I use f = OpenRead["largefile.m"] x = Read[f]; Close[f]; to read in a Mathematica expression from a large file largefile.m ...
9
votes
2answers
227 views
5
votes
2answers
261 views

How to remove an expression from a file?

To read in expressions from a file, one can do ReadList[file, Expression, 3]. How to do the opposite, i.e. remove a number of expressions from a file, modifying ...
20
votes
2answers
2k views

How to save all data in all variables so that loading it is fast?

I've got this CSV file I've imported that has tens of millions of lines in it. It takes around 20 minutes to import. I've been working with it for a while and have the processed data spread out in a ...
3
votes
3answers
326 views

Support for Compressed Sparse Column sparse matrix representation

Is there native support for Compressed Sparse Column (CSC) format for sparse matrices, like importing and manipulating them?
16
votes
1answer
506 views

When importing GIF animation, how to find the correct list of “DisplayDurations”?

When importing GIF animations with variable frame durations, the settings for "DisplayDurations" are not recovered correctly. As an example, I'm using the movie from this earlier question: Here I'm ...
13
votes
3answers
778 views

How to scrape the headlines from New York Times and Wall Street Journal?

How to scrape headlines from New York Times, Wall Street Journal main pages to create datasets similar to this service? Importing HTML from nyt.com (HTML4) results in a ...
6
votes
3answers
208 views

How can I read in specific information from a TeX-file?

I have several TeX-files and they contain information of the form ...
7
votes
1answer
122 views

Using Differences on data: trouble with floats and doubles

Consider the following data set (after I have run FullForm), which is imported from a file (stored typically as 10.040): ...
7
votes
1answer
1k views

Import DCD (CHARMM/NAMD binary trajectory) file

How can one import the coordinates stored in a binary CHARMM/NAMD DCD file? This is the structure of the file: ...
5
votes
2answers
382 views

How to test if file exists and stop

At the moment I'm using > ...
2
votes
1answer
231 views

How to solve issue with exporting / importing data to and from Excel

Background ( simplified ): When I export data to Excel like ...
12
votes
1answer
243 views

Exporting and reimporting a list of lists changes the dimensions

Could anyone explain to me what in the world is going on here and please help me in reading in dummy.dat properly? ...
14
votes
2answers
2k views

Importing a large Excel file

I'm trying to import a simple Excel file containing 94,000 rows and 52 columns of various types, mostly Reals. It's a 38-MB file. I get this super helpful error ...
6
votes
1answer
630 views

Stream CSV files

Ram limitations make import impractical for large CSV files. Is it possible to Stream CSV correctly? ...
19
votes
2answers
1k views

Extracting values from nested rules in JSON data

I have been using Mathematica to analyse some data from the StackExchange API. It is conveniently available in JSON form, which Mathematica interprets as replacement rules. However, some of the rules ...
6
votes
1answer
3k views

How to extract sample values from an imported WAV file?

Suppose I read a file with B = Import["something.wav"] Now how can I obtain a list of sample values?
14
votes
2answers
339 views

How to load thumbnail, not image?

Is there any way to import only the thumbnail of an image instead of the full image? I frequently work with lots of images and it would be nice if the contents of a directory could be quickly ...
4
votes
2answers
4k views

Importing from Excel

I'm trying to import a matrix from Excel to Mathematica. My code is: Import["desktop/stproj.xls", "xls", data, 1] The output is some weird stuff about population ...
8
votes
1answer
446 views

How to properly DumpSave & Get Global`s symbols inside packages while not touching Global context?

For efficiency reasons I prefer to use DumpSave instead of Save. For ease of access I prefer to save symbols in ...
6
votes
3answers
189 views

Organizing similar datasets using DownValues instead of Contexts

In his answer to this question, Leonid helped me find a way to programmatically generate contexts for holding sets of symbols with the same names. Perhaps a simpler way is to use ...
7
votes
1answer
144 views

Organizing similar datasets using Contexts or otherwise

I often import several long time series to analyze something about a city. When I have multiple cities, I use contexts. As a simple example, ...
15
votes
1answer
358 views

Importing Zip files

Mathematica can import zipped files automatically, but when the content of the zip file it's another zipped file, I just get the list of files in the second zip: ...
3
votes
1answer
291 views

Mathematica Import functions fails for data download from website

I am trying to download some data from a website that uses "java enabling" to tell me that I cannot download the data. Using Import[ "website...", "Data"] I get ...
11
votes
6answers
2k views

How do you skip missing data values in a data file?

In scanning through a simple imported file of weather station data, some dates have missing data. What is a test to use to check whether a datum is valid? I tried ...
11
votes
3answers
407 views

Data of badly formatted file

Similar as in this question, I have to read in data from a file (exampleFile.txt) which is not so well formatted. Unfortunately, I cannot find a solution by myself. Since it is a bit complicated to ...
14
votes
1answer
708 views

Release the web camera after using CurrentImage[]

CurrentImage[] will capture an image from the webcam. It is even possible to create a live video in the notebook using ...
49
votes
3answers
5k views

Recovering data points from an image

Unfortunately, some data can only be obtained in picture form (Japanese publications anyone?). Since this cannot be worked with, it has to be converted to a dataset that can; I was wondering whether ...
4
votes
3answers
292 views

Plotting data with exponentials

I have some data with 19000 sublists such as : {" 7.9080000e+01 1.9283193e+04"} Where the first number is the value for variable ...
3
votes
2answers
1k views

Importing .txt or .csv financial data

I am trying to import some financial data exported from Excel. You can find the relevant file here : Text File or CSV File This is what the data looks like in Excel : Using the following code to ...
9
votes
3answers
754 views

Is it possible to import dates and times directly as AbsoluteTime and by pass DateLists?

Date and time calculations are quite slow so I am exploring ways to minimize the number of calculations performed. As an example: ...
1
vote
1answer
414 views

How can I speed up image importing? [closed]

The following examples show how slow it is to load a simple GIF. Can anything be done to make it faster? System #1: Mathematica 8.0.1.0, HP notebook, Win7 64Bit, 2 x 2.4Ghz Intel Core i5, 2GB RAM, w/ ...
17
votes
2answers
383 views

Is Compress[] compatible between different Mathematica versions?

Compress[] was introduced in version 6 of Mathematica. It gives a compressed version of expressions represented as a string of ASCII characters. It preserves more ...
16
votes
2answers
521 views

How to develop an Import/Export converter for Compress[]ed data?

Mathematica has a cross platform data exchange format, WDX. Unfortunately importing or exporting large data from/to WDX is very slow. Using MX files is very fast, but they are not compatible across ...
6
votes
3answers
638 views

Efficient image Import

I have several sets of 20 images I need to import. Each time they have different categories. "Name"<>"Cat"<>"No" Electric B 1 for example. I need to create lists that contain the images ...