Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

13
votes
1answer
422 views

How can I read compressed .Z file automatically by Mathematica?

When I read .Z file by something like jpldatstring = Import["ftp://cddis.gsfc.nasa.gov/products/jpl/2003/jplg0040.04i.Z"] Unfortunately, it turned out that ...
8
votes
1answer
495 views

Converting XMLElement objects to plain text

Short version: How can I convert an XMLElement that represents part of a HTML document to plain text? Long version: The more general problem is extracting ...
10
votes
2answers
314 views

How to extract a specific element from EXIF photographic metadata?

It's now possible to import photographs into Mathematica and import the EXIF data at the same time: i = Import["ExampleData/coneflower.jpg", "ImageWithExif"] ...
8
votes
2answers
6k views

Import selected columns from a tab delimited text file

I have a very huge file (1GB) from which I desire to import only the 5th column. It is a tab delimited text file with a .txt extension. Can you please help me with ...
8
votes
1answer
775 views

How to properly DumpSave & Get Global`s symbols inside packages while not touching Global context?

For efficiency reasons I prefer to use DumpSave instead of Save. For ease of access I prefer to save symbols in ...
6
votes
2answers
330 views

Importing multiple images for machine learning classification

I want to create a ML classifier using images just like this example. I have two classes of images in two different folders. Is there an easier way to import all images from both folders rather than ...
5
votes
2answers
379 views

Sorting a large file

I have some huge logs, which are text files, containing up to a million rows each, and I need them sorted. Because trying to import one obviously makes Mathematica run out of memory, we must rule out ...
15
votes
3answers
657 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
9
votes
2answers
481 views

Convert quoted printable aka PrintableASCII characters to UTF-8

I have several .eml email files to import. Some of them were saved as quoted-printable 7-bit encoded. I skipped through the Mathematica documentation and found ...
6
votes
1answer
571 views

Accessing data on local machine from remote kernel

I have a remote kernel on a server that I would like to use to process some data. The connection works, i.e, I'm able to use the local front end and the remote kernel. However, I cannot figure out how ...
6
votes
1answer
1k views

When do I use Mathematica, when do I use Gephi, and how do I use them together?

Mathematica has great operations for analyzing graphs but when the graphs are a bit bigger, Gephi seems to be more efficient (at least on my computer) at drawing and analyzing, i.e. visualizing the ...
6
votes
1answer
440 views

Operations on online files via public URL access

I need Mathematica to access an online public directory via some URL (fake in this case) http://example.com/images and perform operations similar to those Mathematica can do on OS directories and ...
5
votes
1answer
292 views

How to transform a DumpSaved .mx file to .m file conveniently?

I have many DumpSave-d .mx files under a directory. They are generated under Linux. The .mx ...
5
votes
1answer
980 views

Efficiently exporting many images as a video

I have many .gif images and I would like to export them as a single video file (I prefer .mov, but any video format is fine). But what if I have several tens of megabytes of .gif images? Is it ...
3
votes
2answers
102 views

Most efficient way of splitting a file into length-prefixed blocks

Let's say I have a really big file (>100MB) that consists of length-prefixed blocks, i.e. the first byte of each byte block indicates the length of the rest of the block. This is a simplified example ...
2
votes
2answers
100 views

How to copy Coordinates of “Get Coordinates” in Figure?

I am trying to reproduce the answer here about Recovering data points from an image. I click the figure and press "Get Coordinates". I would like to copy the four coordinates by CTRL-C and then paste ...
13
votes
4answers
409 views

Unencoding encoded URLs in a function

Is there a built-in function for decoding encoded URLs in Mathematica? I would like to convert an encoded URL, for example ...
12
votes
2answers
736 views

Parsing a CSV file using Interpreter

The new Interpreter functionality in version 10 looks like it has the potential to make parsing custom data formats very easy. I'm trying to create a CSV parser. ...
10
votes
1answer
677 views

How to import date (YYYY-MM-DD) data from a file?

I have a file with data in form {date, value}: 2010-05-19 17 2010-05-20 20 ...
9
votes
2answers
1k views

How can I animate a set of points imported from file?

I have an external program which generates a set of data - X points at the time t1, X points at the time t2 and so on. These points are stored in CSV file and I can change the format if necessary. ...
8
votes
1answer
126 views

Is it possible to move .mx file from one server to other?

I have mathematica file in one server along with .mx file. Is there any way to move the .mx file from one server to other such ...
8
votes
1answer
160 views

Organizing similar datasets using Contexts or otherwise

I often import several long time series to analyze something about a city. When I have multiple cities, I use contexts. As a simple example, ...
7
votes
1answer
967 views

Import file from FTP site

I would like to Import files from an FTP site. The code below works on a test site: file = Import["ftp://test:test@ftp.secureftp-test.com/pigs.xml"]; But when I ...
6
votes
1answer
3k views

Reading csv/xlsx data into Mathematica with column headings as list names

It's old question. However, I could not find satisfactory answer. Decided to post this question. I have a .csv file with column heading. How can I import this ...
4
votes
2answers
218 views

How can I encrypt/decrypt a directory?

I'd like to use Mathematica 10's new Encrypt and Decrypt functions to secure files and folders on my mac. Here's what I have so far but something goes wrong along the way, since by the end of this ...
4
votes
1answer
270 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...
4
votes
1answer
211 views

Importing malformed XML: Import[… “XMLObject”] vs ImportString[…, “XML”]

It appears that Import[ url , "XMLObject" ] is more forgiving of improperly formatted XML than ImportString[ string, "XML"]. For ...
4
votes
3answers
3k views

Import specific data from Excel to make a plot

I need some help with this: I have imported data from an Excel file and I need to make a contour plot of it. The data are in table-like form. Here is a sample of the first few 'rows': ...
3
votes
2answers
151 views

Importing CSV with multiline text fields

Here is a valid CSV: a1,b1,c1,d1,"e1 e1 e1" a2,b2,"c2 c2 c2",d2,e2 It has two rows and five columns, and contains two fields with embedded CRLF. (See the CSV ...
3
votes
1answer
427 views

how to delete specific data from text file and then update it

I have a large text file. I read data from text file one at a time and check whether data agree with my condition. If not I want to delete that data in text file and update it. For example say my ...
3
votes
1answer
519 views

Mathematica - set end-of-line characters when importing

I have a plain text file of tab-separated values that was created on a Windows machine. I want to import it into Mathematica on a Mac. ...
1
vote
2answers
1k views

Can I improve the importing speed of Excel data?

I have data in an Excel spreadsheet that has 4 rows and 1,000 columns. I am using Import as follows: ...
9
votes
2answers
292 views

Working with Java hashmaps from Mathematica

I'm using J/Link to access data from an external Java program. The data are represented as a ConcurrentHashMap (see the official specification here). ...
8
votes
1answer
178 views

Reading a HDF file im Mathematica

Bug introduced in 8.0 or earlier and persisting though 10.2 On OS X, the problem is present in 9.0 but not in 10.0, 10.1 or 10.2. I am trying to read a HDF file in Mathematica 9 on OS X and get ...
7
votes
1answer
252 views

Incorrect result while using SemanticImport for CSV file import

I might use this incorrectly. I want to use a .csv file that (as list) looks like: ...
7
votes
2answers
1k views

Why does importing a URL fail the second time and produce a FetchURL::conopen error?

I have Mathematica code that uses Import[] to download URLs. I noticed that first time I use the Import[] command it works. ...
7
votes
3answers
244 views

How can I read in specific information from a TeX-file?

I have several TeX-files and they contain information of the form ...
6
votes
1answer
407 views

How to correctly import data zipped with the “deflate” algorithm?

I'm writing an importer for the GIFTI file format. The details of the format are not particularly important, but the basic idea is that it is a relatively simple XML file which includes binary arrays ...
6
votes
1answer
179 views

Can I read Chinese characters with ReadList correctly?

Just try this example: Export["a.txt", "这乱码问题该怎么解决呢\n***\n怎么解决嘛"] ReadList["a.txt", Record] ...
6
votes
3answers
202 views

Is it possible to Import urls not type “”

These days I often copy urls from Internet Explorer's address bar. When I paste the url into a Mathematica cell it is neither a string nor a expression. ...
6
votes
1answer
299 views

Importing images from the web using ImportString & URLFetch

I can bring images from the web using the Import command like this: Import["http://blog.wolfram.com/wp-content/uploads/2008/06/se-30.jpg"] Sometimes the web ...
4
votes
1answer
318 views

Importing FITS file from HEALPix

I am trying to read FITS file created by HEALPix software. The file contains a spherical map of a scalar field. There is the online support on how to use the file in IDL, but not for Mathematica. I ...
4
votes
2answers
176 views

Workaround for Import not supporting the \leqslant TeX macro

$\{A\leqslant x\leqslant B\}$ Copy the code into Notebook, and you'll lose the less sign, this can be rendered well in ...
3
votes
2answers
224 views

Why are dictionary keys reordered upon JSON import?

After importing a JSON file, Mathematica changed the order of the fields. For example, I downloaded a sample file from: http://api.openweathermap.org/data/2.5/find?lat=55.5&lon=37.5&cnt=10 I ...
3
votes
2answers
1k views

How can I remove text headers when importing a numeric table

I have a file let's say 1.txt having numbers like: fgdgfdg fhdfdh 0.000105533 1.00E-04 0.222281 1.00E-04 0.444456 1.00E-04 0.666631 1.00E-04 asdas ddfdsfsd 0.111193 0.166733 ...
3
votes
2answers
299 views

Speeding up TIF import

After trying to import a $\approx 2$ GB TIF file with the Import command, I noticed that the process takes $\approx 15$ minutes or so, which seems excessive, especially when compared to other software ...
3
votes
2answers
2k views

3D scatter plot from a txt file

I have a txt file with space-delimited x, y, z coordinates of several points. The points are organized into groups of unequal size; the groups are delimited by skipping a line, like this: ...
3
votes
2answers
467 views

Extracting a single layer from a SHP file

I have an ESRI ShapeFile (SHP) that I've loaded. It has a number of layers which I can see the names of by using: Import["file.shp", "LayerNames"] I get ...
3
votes
1answer
733 views

Importing data sets from plain-text files and performing operations on them

I am working with a lot of daily weather files in the txt format. I need to calculate a daily quantity from each of them, e.g., daily light intensity, which ...
2
votes
1answer
59 views

Import tabular text data

I have a text file created with a bash script I wanted to read the data avoiding the first line. I have written few dirty codes, but I wondered if anybody can do it in fewest possible nice one ...