Questions about getting data, files, images etc. from various formats into a form that can be read, tranformed and manipulated by Mathematica

learn more… | top users | synonyms

8
votes
2answers
7k views

Import selected columns from a tab delimited text file

I have a very huge file (1GB) from which I desire to import only the 5th column. It is a tab delimited text file with a .txt extension. Can you please help me with ...
5
votes
2answers
389 views

Sorting a large file

I have some huge logs, which are text files, containing up to a million rows each, and I need them sorted. Because trying to import one obviously makes Mathematica run out of memory, we must rule out ...
15
votes
3answers
669 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
9
votes
2answers
498 views

Convert quoted printable aka PrintableASCII characters to UTF-8

I have several .eml email files to import. Some of them were saved as quoted-printable 7-bit encoded. I skipped through the Mathematica documentation and found ...
7
votes
2answers
512 views

How to speed up interpreting numbers from strings?

In processing a large text file I have batches from ReadList of 100,000 Records of the form. ...
6
votes
1answer
584 views

Accessing data on local machine from remote kernel

I have a remote kernel on a server that I would like to use to process some data. The connection works, i.e, I'm able to use the local front end and the remote kernel. However, I cannot figure out how ...
6
votes
1answer
445 views

Operations on online files via public URL access

I need Mathematica to access an online public directory via some URL (fake in this case) http://example.com/images and perform operations similar to those Mathematica can do on OS directories and ...
6
votes
1answer
1k views

When do I use Mathematica, when do I use Gephi, and how do I use them together?

Mathematica has great operations for analyzing graphs but when the graphs are a bit bigger, Gephi seems to be more efficient (at least on my computer) at drawing and analyzing, i.e. visualizing the ...
5
votes
1answer
999 views

Efficiently exporting many images as a video

I have many .gif images and I would like to export them as a single video file (I prefer .mov, but any video format is fine). But what if I have several tens of megabytes of .gif images? Is it ...
3
votes
2answers
104 views

Most efficient way of splitting a file into length-prefixed blocks

Let's say I have a really big file (>100MB) that consists of length-prefixed blocks, i.e. the first byte of each byte block indicates the length of the rest of the block. This is a simplified example ...
2
votes
2answers
122 views

How to copy Coordinates of “Get Coordinates” in Figure?

I am trying to reproduce the answer here about Recovering data points from an image. I click the figure and press "Get Coordinates". I would like to copy the four coordinates by CTRL-C and then paste ...
13
votes
4answers
416 views

Unencoding encoded URLs in a function

Is there a built-in function for decoding encoded URLs in Mathematica? I would like to convert an encoded URL, for example ...
10
votes
1answer
699 views

How to import date (YYYY-MM-DD) data from a file?

I have a file with data in form {date, value}: 2010-05-19 17 2010-05-20 20 ...
8
votes
1answer
127 views

Is it possible to move .mx file from one server to other?

I have mathematica file in one server along with .mx file. Is there any way to move the .mx file from one server to other such ...
8
votes
1answer
160 views

Organizing similar datasets using Contexts or otherwise

I often import several long time series to analyze something about a city. When I have multiple cities, I use contexts. As a simple example, ...
7
votes
1answer
1k views

Importing specific elements from JSON

I'm having trouble importing JSON in the way I expect, based on my reading of the documentation. The particular issue I'm having is with importing a specific element, or list of elements. From the doc:...
7
votes
1answer
986 views

Import file from FTP site

I would like to Import files from an FTP site. The code below works on a test site: file = Import["ftp://test:test@ftp.secureftp-test.com/pigs.xml"]; But when I ...
6
votes
1answer
3k views

Reading csv/xlsx data into Mathematica with column headings as list names

It's old question. However, I could not find satisfactory answer. Decided to post this question. I have a .csv file with column heading. How can I import this ...
5
votes
1answer
690 views

Open data file from CDFPlayer?

Using a CDF program running in CDF Player, is it possible to use a system dialog box, or even just an Import statement such as Import["*.CSV"], to import data from a text or comma separated file on a ...
4
votes
2answers
227 views

How can I encrypt/decrypt a directory?

I'd like to use Mathematica 10's new Encrypt and Decrypt functions to secure files and folders on my mac. Here's what I have so far but something goes wrong along the way, since by the end of this ...
4
votes
1answer
219 views

Importing malformed XML: Import[… “XMLObject”] vs ImportString[…, “XML”]

It appears that Import[ url , "XMLObject" ] is more forgiving of improperly formatted XML than ImportString[ string, "XML"]. For ...
4
votes
3answers
3k views

Import specific data from Excel to make a plot

I need some help with this: I have imported data from an Excel file and I need to make a contour plot of it. The data are in table-like form. Here is a sample of the first few 'rows': ...
4
votes
1answer
297 views

mathematica use too much memory to Import large dat file

Err... When I try to Import a .dat file by dat = Import["D:\\TIEGCM\\solar_input\\omni_m_all_years.dat", "Table"]; or ...
3
votes
1answer
533 views

Mathematica - set end-of-line characters when importing

I have a plain text file of tab-separated values that was created on a Windows machine. I want to import it into Mathematica on a Mac. ...
3
votes
1answer
447 views

how to delete specific data from text file and then update it

I have a large text file. I read data from text file one at a time and check whether data agree with my condition. If not I want to delete that data in text file and update it. For example say my ...
1
vote
2answers
1k views

Can I improve the importing speed of Excel data?

I have data in an Excel spreadsheet that has 4 rows and 1,000 columns. I am using Import as follows: ...
9
votes
2answers
295 views

Working with Java hashmaps from Mathematica

I'm using J/Link to access data from an external Java program. The data are represented as a ConcurrentHashMap (see the official specification here). ...
8
votes
1answer
184 views

Reading a HDF file im Mathematica

Bug introduced in 8.0 or earlier and persisting though 10.2 On OS X, the problem is present in 9.0 but not in 10.0, 10.1 or 10.2. I am trying to read a HDF file in Mathematica 9 on OS X and get ...
7
votes
3answers
245 views

How can I read in specific information from a TeX-file?

I have several TeX-files and they contain information of the form ...
7
votes
1answer
267 views

Incorrect result while using SemanticImport for CSV file import

I might use this incorrectly. I want to use a .csv file that (as list) looks like: ...
7
votes
2answers
1k views

Why does importing a URL fail the second time and produce a FetchURL::conopen error?

I have Mathematica code that uses Import[] to download URLs. I noticed that first time I use the Import[] command it works. ...
6
votes
1answer
437 views

How to correctly import data zipped with the “deflate” algorithm?

I'm writing an importer for the GIFTI file format. The details of the format are not particularly important, but the basic idea is that it is a relatively simple XML file which includes binary arrays ...
6
votes
3answers
208 views

Is it possible to Import urls not type “”

These days I often copy urls from Internet Explorer's address bar. When I paste the url into a Mathematica cell it is neither a string nor a expression. Import@http://mathematica.stackexchange.com/...
6
votes
1answer
187 views

Can I read Chinese characters with ReadList correctly?

Just try this example: Export["a.txt", "这乱码问题该怎么解决呢\n***\n怎么解决嘛"] ReadList["a.txt", Record] ...
6
votes
1answer
309 views

Importing images from the web using ImportString & URLFetch

I can bring images from the web using the Import command like this: Import["http://blog.wolfram.com/wp-content/uploads/2008/06/se-30.jpg"] Sometimes the web ...
4
votes
2answers
178 views

Workaround for Import not supporting the \leqslant TeX macro

$\{A\leqslant x\leqslant B\}$ Copy the code into Notebook, and you'll lose the less sign, this can be rendered well in ...
4
votes
1answer
333 views

Importing FITS file from HEALPix

I am trying to read FITS file created by HEALPix software. The file contains a spherical map of a scalar field. There is the online support on how to use the file in IDL, but not for Mathematica. I ...
4
votes
2answers
6k views

Importing from Excel

I'm trying to import a matrix from Excel to Mathematica. My code is: Import["desktop/stproj.xls", "xls", data, 1] The output is some weird stuff about population ...
3
votes
2answers
1k views

How can I remove text headers when importing a numeric table

I have a file let's say 1.txt having numbers like: fgdgfdg fhdfdh 0.000105533 1.00E-04 0.222281 1.00E-04 0.444456 1.00E-04 0.666631 1.00E-04 asdas ddfdsfsd 0.111193 0.166733 ...
3
votes
2answers
231 views

Why are dictionary keys reordered upon JSON import?

After importing a JSON file, Mathematica changed the order of the fields. For example, I downloaded a sample file from: http://api.openweathermap.org/data/2.5/find?lat=55.5&lon=37.5&cnt=10 I ...
3
votes
2answers
2k views

3D scatter plot from a txt file

I have a txt file with space-delimited x, y, z coordinates of several points. The points are organized into groups of unequal size; the groups are delimited by skipping a line, like this: ...
3
votes
1answer
747 views

Importing data sets from plain-text files and performing operations on them

I am working with a lot of daily weather files in the txt format. I need to calculate a daily quantity from each of them, e.g., daily light intensity, which ...
3
votes
2answers
477 views

Extracting a single layer from a SHP file

I have an ESRI ShapeFile (SHP) that I've loaded. It has a number of layers which I can see the names of by using: Import["file.shp", "LayerNames"] I get ...
3
votes
2answers
304 views

Speeding up TIF import

After trying to import a $\approx 2$ GB TIF file with the Import command, I noticed that the process takes $\approx 15$ minutes or so, which seems excessive, especially when compared to other software ...
2
votes
1answer
613 views

How to work with 3D images of STL format

After importing a 3D image of STL format, my possibilities in regard to "3D Volumetric Image Processing" seems rather limited. All examples are given with TIFF files. Is there a way to convert from ...
2
votes
1answer
277 views

importing large data files with a constraint

As I have learned in this community, one way to import a hefty data files (e.g. millions of rows of {x1, y1} data) if you don't care about the individual elements (maybe just want to obtain some ...
1
vote
1answer
2k views

Importing a matrix from a text file

I have a $4 \times N$ matrix in a .txt file of the form: -17.071 137.567 16.554 0.65 -16.873 136.138 16.934 0.7 -15.397 135.738 16.894 0.7 -14.7 135.982 15.9 0.6 ...
1
vote
1answer
1k views

How to set path of IMPORT from excel into Mathematica?

I was able to IMPORT from excel data at the start of my project when both the .nb and EXCEL XLS files were sitting in MyDocument (or some other location), but upon moving both files into my Dropbox ...
1
vote
0answers
55 views

How can I use Mathematica to get the translated result from the Google Translate webpage? [duplicate]

I would like to use Mathematica to automatically translate some input texts, from one language to the other one, then apply some manipulation of a string on the text. I know how to realize the ...
1
vote
1answer
259 views

How do I upload a graph as an adjacency list and find the betweenness centrality?

I have an undirected simple graph in a .txt file formatted as an adjacency list like this: 100 200 200 300 300 400 500 600 700 800 900 ... Every number is a node ...