# Tag Info

## Hot answers tagged data-format

25

For a one-off read you can Skip a number of records: str = OpenRead["test.tsv"]; Skip[str, Record, n - 1]; data = ReadList[str, {Record, Number, Record}, 100, RecordSeparators -> {"\t", "\n"}]; Close[str]; If you will be reading from the same file many times, it may be worth building an index you can use with SetStreamPosition str = ...

14

Comment This was originally answered on Oct 2, 2012 using V8. The performance can be dramatically improved using V9's URLFetchAsynchronous, as now shown below. Fortunately, we needn't download all the tiles at once. We can use Dynamic to set up a little pan-and-zoom explorer. The first load takes a bit and zooming out takes a bit. Panning and zooming ...

12

Mathematica is not really a reference in digital terrain models, and, there are very powerful software packages to deal with geographical information. But... where would be the fun... If the information in the files includes a digital terrain model (DTM), with the surface of the terrain defined by triangular faces (regular or not), then you can easily ...

10

You should be able to use ReadList on the string contents of each sublist. Here I'm just creating a small list containing three elements identical to the one you provided. The result can be plotted using ListPlot for example. In[20]:= in = {{" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" ...

9

Time for regular expressions to save the day once again! line = "³00001³96.15E+01³76186.E-01³-3894.E-04³ 2726.E-04³\ 0005.05³ Si ³52049.E-04³000000001.000³ ³ ³" (* Matches one number in the horrible log file: *) regex = RegularExpression[ "(\\d+(\\.\\d*(e[+-]\\d+)?)?)" ] -> "$1"; matches = StringCases[line, regex, IgnoreCase ... 9 I think, using contexts here is a sensible suggestion, particularly because you want to use several variables. One possible alternative is to set up a struct-like data structure, where encapsulation mechanism is based on Module-generated persistent variables. There were many discussions related to emulation of structs in Mathematica, but, given the ... 9 This should give you back most of your work: code = Import[ "https://bitbucket.org/ricopicone/corruptmathematicanotebook/raw/master/CMN.nb", "NB"]; CreateDocument[ code /. Cell[BoxData[(Graphics3DBox | GraphicsBox)[___]], ___] :> Cell[TextData[ StyleBox["Here was a graphics", FontColor -> RGBColor[1, 0, 0]]], "Text"]] 8 The documentation is misleading here. On one hand, the only export option is "Append" which can be found under the Options tab. On the other hand, the general documentation reads I really wonder, why it is necessary to put Import only behind an option value when "DataEncoding" isn't an export option at all. Anyway, I have the same behaviour in MacOSX ... 8 As Szabolcs points out, this may be imported by Import. Here's one way to discover this: assign expr = " 7.9080000e+01 1.9283193e+04", then use ImportString on it with all possible import formats, discard those that return$Failed, and look at the results: Grid[ DeleteCases[ Quiet[{#, ImportString[expr, #]}] & /@ $ImportFormats, x_List /; ... 8 Here are two approaches. We'll create a second dataset by shifting the given data by two months: blossom = {{4, 3}, {4, 22}, {4, 15}, {4, 2}, {4, 18}, {4, 20}, {4, 12}, {3, 30}, {4, 4}, {4, 24}, {4, 26}, {3, 4}, {4, 26}, {4, 13}, {5, 1}, {4, 4}, {4, 8}, {4, 18}, {4, 9}, {4, 19}, {4, 10}, {4, 20}, {4, 3}, {4, 4}, {3, 21}, {4, 19}, {4, 15}, ... 7 Maybe you could do something like this: First the data is imported as a list of strings. import = Import["exampleFile.txt", "List"]; pos is the position of all lines containing the substring "Ion". The data of interest is located 3 lines below the lines containing "Ion". Therefore we omit the last four lines of import when searching. pos = ... 7 I echo Szabolcs's comment that you should probably have done this on acquisition, but now you could use this: dat = { {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"} }; ImportString@ExportString[dat, "Table"] {{79.08, 19283.2}, {79.08, 19283.2}, {79.08, 19283.2}} 7 Here is a solution for Linux/( ? Mac ? ) uses, or Windows users with a Linux toolkit. To read line 23681 to 23781, use the following variants of head and tail. data=Import["!head -23781 /tmp/data.txt | tail -n 100", "Text"]; 7 If I am not mistaken the 212 format is storing two signals and each 3-byte group provides a 12-bit reading of that pair. Importing with Byte isn't too bad for this set: data = Import["http://physionet.org/physiobank/database/mitdb/100.dat", "Byte"] ~ Partition ~ 3 Then two helpers to parse it: twosComplement[a_, n_] := If[a < 2^(n - 1), a, a - 2^n] ... 6 Import is very well able to handle this format. As a demonstration I use its nephew ImportString to deal with the few lines from your example: data = ImportString[ "28/04/2013 20:01:36.18 2.5013E-2 W 28/04/2013 20:01:36.26 2.5013E-2 W 28/04/2013 20:01:36.32 2.5013E-2 W 28/04/2013 20:01:36.35 2.5011E-2 W 28/04/2013 20:01:36.48 ... 6 Preface: Since you want to convert other C++ classes to MTensor I have to tell you, that I don't see a possibility for that, because I haven't found how to tell the Wolfram Library to take a chunk of already allocated data and use it for an MTensor. What I do is the other way around. I use the internal data of an MTensor as underlying array for my ... 6 data = Table[(i + j) i (i - j), {i, 1, 2}, {j, 1, 22}]; The first step is to define a function to break up each row in the data into lines not exceeded the specified maximum line length. prep[row_, fieldWidth_, lineLength_] := Module[{items = Floor[lineLength/fieldWidth]}, Partition[row, items, items, 1, {}]] The second step is to do the ... 5 The first Google hit for "compressed sparse column" is this page, which explains that it's also called the Harwell-Boeing format. If this is what you need, this format is supported by Mathematica. 5 The MX format (that is written by DumpSave) is not portable between different computer systems. One can't count on it being compatible either between versions, different OSs or different architectures (32 vs 64 bit). I recommend that you use Compress and export the results as a string instead, as I described here. This is not as fast as saving to MX, but ... 5 There's no easy way, it's a custom script that assembles the image out of individual slices, and it's written by someone who clearly didn't intend anyone to read it again (including himself). Reverse engineering. The script responsible is http://imgs.xkcd.com/clickdrag/1110.js, the image to be displayed is assembled in line 86 ($image=...). Scanning the ...

5

I recently had a similar task decoding 12-bit binary 24-channel electroencephalogram data files and found that bit shifting and masking with BitAnd are the way to go. The specific approach depends on file structure, endianness and channel number. In the case of this 'format 212' binary data file containing two interlaced signals, the following procedure ...

4

You can download all the original tiles using the following functions. 404 and file not founds are handled gracefully. I'm avoiding displaying to the FE so as to lower the chances of crashing. url[n1_Integer, d1_String, n2_Integer, d2_String] := "http://imgs.xkcd.com/clickdrag/" <> ToString@n1 <> d1 <> ToString@n2 <> d2 <> ...

4

Carl has shown a way to write to the file what you want, but I would suggest to not even put the terms= into the file: you can read the expression and set it to a variable when reading it. It will also get rid of all kinds of complications that occure if you want to write partially evaluated expressions. This approach lets you choose the name of the variable ...

4

Here is another option that should be pretty fast: Internal`StringToDouble /@ StringSplit[#, "\.b3"][[3 ;; 5]] & /@ Rest @ FindList["exampleFile.txt", "\n\.b3"] Rest is needed to skip one line in the header.

4

Since you requested performance I would avoid Import and DateList and use ReadList and AbsoluteTime. format = {Number, Character, Number, Character, Number, Number, Character, Number, Character, Number, Number, Word}; data = {AbsoluteTime[{#5, #3, #1, #6, #8, #10}], ##11} & @@@ ReadList["data.txt", format]; "data.txt" is of course your data ...

4

Change this {x_, y_} -> x + I y to {x_?NumberQ, y_?NumberQ} -> x + I y

4

In version 7 halirutan's export method does not produce a file that is recognized by Import. However, one can write: Export["matrix2.h5.gz", datapourrie, {"GZIP", "HDF5"}] And then: d2 = Import["matrix2.h5.gz", {"Datasets", "/Dataset1"}]; datapourrie == d2 True

4

According to the comments you have several options: Use Import and Export to work with Excel files (noted: this is not straightforward with problems mentioned in the comments). Use this answer to manipulate the Excel file like it is done in VB. Using Leonid's SQL DSL as a way to get database like syntax while keeping all the data in Mathematica lists. I ...

4

For large files using ReadList is the way to go. So if we assume the name of your file is "file.txt", then the following should be efficient: StringReplace[#, {"e" -> "*^", "j" -> "I"}] & /@ ReadList["file.txt", Word, WordSeparators -> {"(", ")", " "}, RecordLists -> True] // ToExpression

Only top voted, non community-wiki answers of a minimum length are eligible