Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

14
votes
2answers
619 views

How to monitor the progress of Read?

I use f = OpenRead["largefile.m"] x = Read[f]; Close[f]; to read in a Mathematica expression from a large file largefile.m ...
7
votes
1answer
258 views

Assign the contents of a “Copy” operation programmatically to a variable?

While an earlier question Cut and paste data from a spreadsheet nominally addresses the issue, a year has past since its OP asked it and I hope that others have come across simpler or more direct ...
6
votes
1answer
293 views

Reading specific entries from a CSV file

This question is related to the following: Speeding up Import and Export in CSV format Stream CSV files Reading periodic elements from a large file I have a set of very large CSV files (a few ...
6
votes
1answer
895 views

Stream CSV files

Ram limitations make import impractical for large CSV files. Is it possible to Stream CSV correctly? ...
19
votes
1answer
773 views

When importing GIF animation, how to find the correct list of “DisplayDurations”?

When importing GIF animations with variable frame durations, the settings for "DisplayDurations" are not recovered correctly. As an example, I'm using the movie from this earlier question: Here I'm ...
15
votes
1answer
336 views

Why does ImportString[“1c”, “Table”] eat the letter c?

I have some whitespace-separated matrix data that I read with Import[..., "Table"]. The data contained mixed strings and numbers (the strings are for row and ...
4
votes
2answers
399 views

How to save a web page in mathematica?

For a given web page http://www.nytimes.com/, is it possible to save it as : first: only HTML(within css) second: more elements(include pictures and so on) just like what a browser can do. ...
3
votes
2answers
2k views

Importing .txt or .csv financial data

I am trying to import some financial data exported from Excel. You can find the relevant file here : Text File or CSV File This is what the data looks like in Excel : Using the following code to ...
16
votes
1answer
483 views

Importing Zip files

Mathematica can import zipped files automatically, but when the content of the zip file it's another zipped file, I just get the list of files in the second zip: ...
7
votes
3answers
939 views

Efficient image Import

I have several sets of 20 images I need to import. Each time they have different categories. "Name"<>"Cat"<>"No" Electric B 1 for example. I need to create lists that contain the images ...
5
votes
1answer
133 views

Is there a way to embed a data set in a notebook?

I have a data set (currently a table in a .csv file). I would like to share this notebook with colleagues either as a notebook, or (even better) as a CDF document. Is there a way to embed the data ...
4
votes
2answers
2k views

Reading in scientific notation from C++ to Mathematica

With a text file with entries formatted as (output from C++): f[38.67] = -2.5387862698183892298317350539374412777263289550697e-05; ⋮ Is there a way to read this ...
5
votes
1answer
2k views

Combine multiple PDF files into one document

I have multiple single page PDFs I'd like to join together into a single file. (Obviously I mean the file format and not the probability distributions here!) I'm aware of free online services such as ...
4
votes
1answer
784 views

ReadList with header and foot lines

I want to do some large data analysis, and i want to be able to import files in a fast way. I tried Import, but it is quite slow; ReadList takes half the time. This is the code I was using: ...
4
votes
2answers
5k views

Importing from Excel

I'm trying to import a matrix from Excel to Mathematica. My code is: Import["desktop/stproj.xls", "xls", data, 1] The output is some weird stuff about population ...
3
votes
1answer
172 views

how can i associate the headings of columns/rows with content of itself and then recall what i want after i've imported the excel file in mathematica

Hello, i make another try... I have to import a file like this below (it is an excel file, i show you just the beginnig. it should have thousend of rows and columns). After i've imported the file i ...
3
votes
0answers
210 views

Errors importing UTF8 file to mathematica

I'm trying to import this file: Import["https://dl.dropboxusercontent.com/u/35192406/jarlaxxe_tweets.\ csv", CharacterEncoding -> "UTF8"] But I am getting ...
1
vote
1answer
1k views

How to set path of IMPORT from excel into Mathematica?

I was able to IMPORT from excel data at the start of my project when both the .nb and EXCEL XLS files were sitting in MyDocument (or some other location), but upon moving both files into my Dropbox ...
5
votes
0answers
1k views

Import Large Excel File

Currently, I am working on a large time series dataset, which contains 928,586 rows and 35 time intervals in columns, in excel format (.xlsx) and using Import to ...
4
votes
1answer
318 views

How can I colorize an imported model?

How can I colorize an imported model from for example 3ds max? I'm trying to find the way to colorize a 3D plot from an imported model with something like this: ...
3
votes
3answers
727 views

Import data from first/last 100 rows of a file

If one wants import data from the first three rows of a file "myfile.data", one can use the command Import["myfile.data",{"Data",{1,2,3},All}]; However, if I ...
3
votes
1answer
101 views

Registering/detecting an importer by file name extension

I've developed an importer and exporter for FreeSurfer's MGH/MGZ file format; the specific format isn't important, but I was ...
3
votes
1answer
925 views

How do I read a CSV file and remove comments from it

I have very recently started using Mathematica 9.0 Home Edition and am trying to understand it as best I can so please be tolerant ;). I have a file of data (sugar.csv) which contains blood sugar ...
3
votes
1answer
498 views

Running external pdflatex command and importing the generated PDF document

There have been a couple of questions about incorporating labels and equations that were typeset in $\LaTeX$ into a Mathematica notebook. Ideally, this requires running ...
2
votes
2answers
180 views

Animation of moving rectangles in 2D from data file

I am not very familiar with mathematica but I know the basics so I thought I give it a try. For a project I have to animate rectangles moving around in 2D (later also 3D). The data come from a C ...
2
votes
1answer
670 views

Making a ListVectorPlot3D from data in an external file

I have a .dat file with six columns: {x y z vx vy vz}, where x, y, z are space coordinates and vx, vy, vz are the vector components. How can I use this file to get ...
2
votes
4answers
3k views

Plotting a large number of data points

I would like to know how to plot when I have a large number of points (arround 600) in a specific format: ...
1
vote
1answer
127 views

Import camera parameters and data from *.sif image files? [closed]

By default, Andor's line of scientific cameras saves data in an apparently undocumented *.sif format. Are there any Mathematica packages or code that can help import these data? I've inherited an ...
1
vote
2answers
459 views

Import large amount of data with time stamps

I have a large data file (150 000 lines) which I want to import in Mathematica. The first few lines look like this ...
-3
votes
1answer
859 views

import Excel sheets by name

I'm importing data from Excel. It appears that the sheet number needed in the Import statement is taken from the order that the sheets appear in the open Excel file. I need to import 2 of the sheets ...
36
votes
3answers
3k views

How to do http POST in Mathematica?

The Import command lets you do a HTTP GET from a URL, like this Import @ "http://merkosoncampus.com/wp-content/uploads/2011/05/higgs-boson.jpg" This sends an ...
23
votes
4answers
4k views

Importing a large Excel file

I'm trying to import a simple Excel file containing 94,000 rows and 52 columns of various types, mostly Reals. It's a 38-MB file. I get this super helpful error ...
24
votes
1answer
3k views

Programmatically retrieving e-mails from a remote server

Mathematica has a function for sending mail (SendMail) and can import the MBOX format. But what I'd like to do is ...
19
votes
2answers
864 views

Why does ImageData need four times more memory?

Background We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
12
votes
3answers
4k views

Excel Headers --> Mathematica

I have a large Excel .xls file. Miniature version shown below: (formatting should appear as two columns...) ...
8
votes
1answer
233 views

How to use RunProcess to import external function?

I have a functions written in an external language called GO. How can I quickly load them into the kernel for use as Mathematica functions? The two ways I've tried are with MathLink and LibraryLink ...
10
votes
4answers
566 views

Exporting data extensions in fits file

Context In Astronomy the de facto standard for images and data is FITS. I would like to export heterogenous data sets into a single fits file. Attempt I am ...
7
votes
1answer
444 views

Import iTunes xml data and convert it into a dataset or table

I have an xml file with song data from iTunes (https://dl.dropboxusercontent.com/u/1012958/iTunes%20Library.xml) and I'd like to import it to Mathematica to do some statistics. I get an XMLObject like ...
8
votes
1answer
471 views

Converting XMLElement objects to plain text

Short version: How can I convert an XMLElement that represents part of a HTML document to plain text? Long version: The more general problem is extracting ...
10
votes
2answers
299 views

How to extract a specific element from EXIF photographic metadata?

It's now possible to import photographs into Mathematica and import the EXIF data at the same time: i = Import["ExampleData/coneflower.jpg", "ImageWithExif"] ...
8
votes
2answers
5k views

Import selected columns from a tab delimited text file

I have a very huge file (1GB) from which I desire to import only the 5th column. It is a tab delimited text file with a .txt extension. Can you please help me with ...
6
votes
2answers
284 views

Importing multiple images for machine learning classification

I want to create a ML classifier using images just like this example. I have two classes of images in two different folders. Is there an easier way to import all images from both folders rather than ...
15
votes
3answers
627 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
13
votes
4answers
1k views

Import[ ] drops leading zeros from data when importing a CSV file

I need to load data from the CSV file shown below. Import[ ] drops the leading zeros in column 3 when it imports the CSV file, but they are security identifiers, and I need them. ReadList[ ] ...
9
votes
2answers
187 views

Inconsistent behavior of undocumented Encode->Import->StringToStream->Get on password-locked stream

Let's save a definition. Encode it with password/key and Get it again. We will not use Get ...
8
votes
1answer
715 views

How to properly DumpSave & Get Global`s symbols inside packages while not touching Global context?

For efficiency reasons I prefer to use DumpSave instead of Save. For ease of access I prefer to save symbols in ...
6
votes
1answer
532 views

Accessing data on local machine from remote kernel

I have a remote kernel on a server that I would like to use to process some data. The connection works, i.e, I'm able to use the local front end and the remote kernel. However, I cannot figure out how ...
6
votes
1answer
1k views

When do I use Mathematica, when do I use Gephi, and how do I use them together?

Mathematica has great operations for analyzing graphs but when the graphs are a bit bigger, Gephi seems to be more efficient (at least on my computer) at drawing and analyzing, i.e. visualizing the ...
6
votes
1answer
431 views

Operations on online files via public URL access

I need Mathematica to access an online public directory via some URL (fake in this case) http://example.com/images and perform operations similar to those Mathematica can do on OS directories and ...
5
votes
2answers
354 views

Sorting a large file

I have some huge logs, which are text files, containing up to a million rows each, and I need them sorted. Because trying to import one obviously makes Mathematica run out of memory, we must rule out ...