# Tag Info

## Hot answers tagged data-format

33

For a one-off read you can Skip a number of records: str = OpenRead["test.tsv"]; Skip[str, Record, n - 1]; data = ReadList[str, {Record, Number, Record}, 100, RecordSeparators -> {"\t", "\n"}]; Close[str]; If you will be reading from the same file many times, it may be worth building an index you can use with SetStreamPosition str = OpenRead["test.tsv"]...

16

Comment This was originally answered on Oct 2, 2012 using V8. The performance can be dramatically improved using V9's URLFetchAsynchronous, as now shown below. Fortunately, we needn't download all the tiles at once. We can use Dynamic to set up a little pan-and-zoom explorer. The first load takes a bit and zooming out takes a bit. Panning and zooming ...

15

Mathematica is not really a reference in digital terrain models, and, there are very powerful software packages to deal with geographical information. But... where would be the fun... If the information in the files includes a digital terrain model (DTM), with the surface of the terrain defined by triangular faces (regular or not), then you can easily ...

12

The documentation is misleading here. On one hand, the only export option is "Append" which can be found under the Options tab. On the other hand, the general documentation reads I really wonder, why it is necessary to put Import only behind an option value when "DataEncoding" isn't an export option at all. Anyway, I have the same behaviour in MacOSX ...

11

You should be able to use ReadList on the string contents of each sublist. Here I'm just creating a small list containing three elements identical to the one you provided. The result can be plotted using ListPlot for example. In[20]:= in = {{" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" 7....

10

Time for regular expressions to save the day once again! line = "³00001³96.15E+01³76186.E-01³-3894.E-04³ 2726.E-04³\ 0005.05³ Si ³52049.E-04³000000001.000³ ³ ³" (* Matches one number in the horrible log file: *) regex = RegularExpression[ "(\\d+(\\.\\d*(e[+-]\\d+)?)?)" ] -> "$1"; matches = StringCases[line, regex, IgnoreCase -&... 10 Here is a solution for Linux/( ? Mac ? ) uses, or Windows users with a Linux toolkit. To read line 23681 to 23781, use the following variants of head and tail. data=Import["!head -23781 /tmp/data.txt | tail -n 100", "Text"]; 9 I think, using contexts here is a sensible suggestion, particularly because you want to use several variables. One possible alternative is to set up a struct-like data structure, where encapsulation mechanism is based on Module-generated persistent variables. There were many discussions related to emulation of structs in Mathematica, but, given the ... 9 As Szabolcs points out, this may be imported by Import. Here's one way to discover this: assign expr = " 7.9080000e+01 1.9283193e+04", then use ImportString on it with all possible import formats, discard those that return$Failed, and look at the results: Grid[ DeleteCases[ Quiet[{#, ImportString[expr, #]}] & /@ $ImportFormats, x_List /; (Last[... 9 This should give you back most of your work: code = Import[ "https://bitbucket.org/ricopicone/corruptmathematicanotebook/raw/master/CMN.nb", "NB"]; CreateDocument[ code /. Cell[BoxData[(Graphics3DBox | GraphicsBox)[___]], ___] :> Cell[TextData[ StyleBox["Here was a graphics", FontColor -> RGBColor[1, 0, 0]]], "Text"]] 8 Preface: Since you want to convert other C++ classes to MTensor I have to tell you, that I don't see a possibility for that, because I haven't found how to tell the Wolfram Library to take a chunk of already allocated data and use it for an MTensor. What I do is the other way around. I use the internal data of an MTensor as underlying array for my own ... 8 Maybe you could do something like this: First the data is imported as a list of strings. import = Import["exampleFile.txt", "List"]; pos is the position of all lines containing the substring "Ion". The data of interest is located 3 lines below the lines containing "Ion". Therefore we omit the last four lines of import when searching. pos = Flatten[... 8 One workaround is to compress the HDF5 file after it has been exported from Mathematica, using the HDF5 command line tools. Note: on OS X the command line tools can be easily installed using MacPorts using port install h5utils. The command to recompress the data is h5repack -v -f GZIP=1 infile.h5 outfile.h5 This can indeed achieve a significant reduction ... 7 I echo Szabolcs's comment that you should probably have done this on acquisition, but now you could use this: dat = { {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"} }; ImportString@ExportString[dat, "Table"] {{79.08, 19283.2}, {79.08, 19283.2}, {79.08, 19283.2}} 7 If I am not mistaken the 212 format is storing two signals and each 3-byte group provides a 12-bit reading of that pair. Importing with Byte isn't too bad for this set: data = Import["http://physionet.org/physiobank/database/mitdb/100.dat", "Byte"] ~ Partition ~ 3 Then two helpers to parse it: twosComplement[a_, n_] := If[a < 2^(n - 1), a, a - 2^n] ... 7 New solution using RegisterFormat from Wolfram Function Repository: ResourceFunction["RegisterFormat"]["BVH", <|"Extension" -> "bvh"|>] Original Post: Copy/paste from the excellent BVH package by @Sjoerd C. de Vries: The code in this question (Registering/detecting an importer by file name extension) did not work. Neither did the answer. Wolfram ... 6 The first Google hit for "compressed sparse column" is this page, which explains that it's also called the Harwell-Boeing format. If this is what you need, this format is supported by Mathematica. 6 The MX format (that is written by DumpSave) is not portable between different computer systems. One can't count on it being compatible either between versions, different OSs or different architectures (32 vs 64 bit). I recommend that you use Compress and export the results as a string instead, as I described here. This is not as fast as saving to MX, but ... 6 In version 7 halirutan's export method does not produce a file that is recognized by Import. However, one can write: Export["matrix2.h5.gz", datapourrie, {"GZIP", "HDF5"}] And then: d2 = Import["matrix2.h5.gz", {"Datasets", "/Dataset1"}]; datapourrie == d2 True 6 Import is very well able to handle this format. As a demonstration I use its nephew ImportString to deal with the few lines from your example: data = ImportString[ "28/04/2013 20:01:36.18 2.5013E-2 W 28/04/2013 20:01:36.26 2.5013E-2 W 28/04/2013 20:01:36.32 2.5013E-2 W 28/04/2013 20:01:36.35 2.5011E-2 W 28/04/2013 20:01:36.48 2.5011E-... 6 I recently had a similar task decoding 12-bit binary 24-channel electroencephalogram data files and found that bit shifting and masking with BitAnd are the way to go. The specific approach depends on file structure, endianness and channel number. In the case of this 'format 212' binary data file containing two interlaced signals, the following procedure ... 6 How about: data = {"0x1", "0xff", "0x1230abde"}; FromDigits[StringTrim[#, "0x"], 16] & /@ data {1, 255, 305179614} 6 data = Table[(i + j) i (i - j), {i, 1, 2}, {j, 1, 22}]; The first step is to define a function to break up each row in the data into lines not exceeded the specified maximum line length. prep[row_, fieldWidth_, lineLength_] := Module[{items = Floor[lineLength/fieldWidth]}, Partition[row, items, items, 1, {}]] The second step is to do the breaking.... 5 There's no easy way, it's a custom script that assembles the image out of individual slices, and it's written by someone who clearly didn't intend anyone to read it again (including himself). Reverse engineering. The script responsible is http://imgs.xkcd.com/clickdrag/1110.js, the image to be displayed is assembled in line 86 ($image=...). Scanning the ...

5

You can download all the original tiles using the following functions. 404 and file not founds are handled gracefully. I'm avoiding displaying to the FE so as to lower the chances of crashing. url[n1_Integer, d1_String, n2_Integer, d2_String] := "http://imgs.xkcd.com/clickdrag/" <> ToString@n1 <> d1 <> ToString@n2 <> d2 <> ...

5

Here is another option that should be pretty fast: Internal`StringToDouble /@ StringSplit[#, "\.b3"][[3 ;; 5]] & /@ Rest @ FindList["exampleFile.txt", "\n\.b3"] Rest is needed to skip one line in the header.

5

Change this {x_, y_} -> x + I y to {x_?NumberQ, y_?NumberQ} -> x + I y

4

You can use set stream position which sets the current point in an open stream. Then you can use Read which reads one expression from an input stream, and returns the expression with byte, character, expression, Number, Real, String or Word options. Something like: Edit 1 with ReadList n = 4; str = OpenRead["data.txt"]; Skip[str, Record, n - 1] ReadList[...

4

I would convert your file first to CSV format. Then, there are a number of options for importing CSV files. If Import is too slow, you can try something similar to what is described here or here. The method described in the former link may require custom code, while the first method of the two described in the latter link is quite general and should work ...

Only top voted, non community-wiki answers of a minimum length are eligible