# Tag Info

21

For a one-off read you can Skip a number of records: str = OpenRead["test.tsv"]; Skip[str, Record, n - 1]; data = ReadList[str, {Record, Number, Record}, 100, RecordSeparators -> {"\t", "\n"}]; Close[str]; If you will be reading from the same file many times, it may be worth building an index you can use with SetStreamPosition str = ...

13

Comment This was originally answered on Oct 2, 2012 using V8. The performance can be dramatically improved using V9's URLFetchAsynchronous, as now shown below. Fortunately, we needn't download all the tiles at once. We can use Dynamic to set up a little pan-and-zoom explorer. The first load takes a bit and zooming out takes a bit. Panning and zooming ...

12

Mathematica is not really a reference in digital terrain models, and, there are very powerful software packages to deal with geographical information. But... where would be the fun... If the information in the files includes a digital terrain model (DTM), with the surface of the terrain defined by triangular faces (regular or not), then you can easily ...

10

You should be able to use ReadList on the string contents of each sublist. Here I'm just creating a small list containing three elements identical to the one you provided. The result can be plotted using ListPlot for example. In[20]:= in = {{" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" ...

9

I think, using contexts here is a sensible suggestion, particularly because you want to use several variables. One possible alternative is to set up a struct-like data structure, where encapsulation mechanism is based on Module-generated persistent variables. There were many discussions related to emulation of structs in Mathematica, but, given the ...

9

Time for regular expressions to save the day once again! line = "³00001³96.15E+01³76186.E-01³-3894.E-04³ 2726.E-04³\ 0005.05³ Si ³52049.E-04³000000001.000³ ³ ³" (* Matches one number in the horrible log file: *) regex = RegularExpression[ "(\\d+(\\.\\d*(e[+-]\\d+)?)?)" ] -> "$1"; matches = StringCases[line, regex, IgnoreCase ... 8 As Szabolcs points out, this may be imported by Import. Here's one way to discover this: assign expr = " 7.9080000e+01 1.9283193e+04", then use ImportString on it with all possible import formats, discard those that return$Failed, and look at the results: Grid[ DeleteCases[ Quiet[{#, ImportString[expr, #]}] & /@ $ImportFormats, x_List /; ... 8 Here are two approaches. We'll create a second dataset by shifting the given data by two months: blossom = {{4, 3}, {4, 22}, {4, 15}, {4, 2}, {4, 18}, {4, 20}, {4, 12}, {3, 30}, {4, 4}, {4, 24}, {4, 26}, {3, 4}, {4, 26}, {4, 13}, {5, 1}, {4, 4}, {4, 8}, {4, 18}, {4, 9}, {4, 19}, {4, 10}, {4, 20}, {4, 3}, {4, 4}, {3, 21}, {4, 19}, {4, 15}, ... 7 I echo Szabolcs's comment that you should probably have done this on acquisition, but now you could use this: dat = { {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"}, {" 7.9080000e+01 1.9283193e+04"} }; ImportString@ExportString[dat, "Table"] {{79.08, 19283.2}, {79.08, 19283.2}, {79.08, 19283.2}} 7 Maybe you could do something like this: First the data is imported as a list of strings. import = Import["exampleFile.txt", "List"]; pos is the position of all lines containing the substring "Ion". The data of interest is located 3 lines below the lines containing "Ion". Therefore we omit the last four lines of import when searching. pos = ... 6 The documentation is misleading here. On one hand, the only export option is "Append" which can be found under the Options tab. On the other hand, the general documentation reads I really wonder, why it is necessary to put Import only behind an option value when "DataEncoding" isn't an export option at all. Anyway, I have the same behaviour in MacOSX ... 6 Import is very well able to handle this format. As a demonstration I use its nephew ImportString to deal with the few lines from your example: data = ImportString[ "28/04/2013 20:01:36.18 2.5013E-2 W 28/04/2013 20:01:36.26 2.5013E-2 W 28/04/2013 20:01:36.32 2.5013E-2 W 28/04/2013 20:01:36.35 2.5011E-2 W 28/04/2013 20:01:36.48 ... 6 If I am not mistaken the 212 format is storing two signals and each 3-byte group provides a 12-bit reading of that pair. Importing with Byte isn't too bad for this set: data = Import["http://physionet.org/physiobank/database/mitdb/100.dat", "Byte"] ~ Partition ~ 3 Then two helpers to parse it: twosComplement[a_, n_] := If[a < 2^(n - 1), a, a - 2^n] ... 5 The first Google hit for "compressed sparse column" is this page, which explains that it's also called the Harwell-Boeing format. If this is what you need, this format is supported by Mathematica. 5 Preface: Since you want to convert other C++ classes to MTensor I have to tell you, that I don't see a possibility for that, because I haven't found how to tell the Wolfram Library to take a chunk of already allocated data and use it for an MTensor. What I do is the other way around. I use the internal data of an MTensor as underlying array for my ... 5 The MX format (that is written by DumpSave) is not portable between different computer systems. One can't count on it being compatible either between versions, different OSs or different architectures (32 vs 64 bit). I recommend that you use Compress and export the results as a string instead, as I described here. This is not as fast as saving to MX, but ... 5 There's no easy way, it's a custom script that assembles the image out of individual slices, and it's written by someone who clearly didn't intend anyone to read it again (including himself). Reverse engineering. The script responsible is http://imgs.xkcd.com/clickdrag/1110.js, the image to be displayed is assembled in line 86 ($image=...). Scanning the ...

5

I recently had a similar task decoding 12-bit binary 24-channel electroencephalogram data files and found that bit shifting and masking with BitAnd are the way to go. The specific approach depends on file structure, endianness and channel number. In the case of this 'format 212' binary data file containing two interlaced signals, the following procedure ...

4

You can download all the original tiles using the following functions. 404 and file not founds are handled gracefully. I'm avoiding displaying to the FE so as to lower the chances of crashing. url[n1_Integer, d1_String, n2_Integer, d2_String] := "http://imgs.xkcd.com/clickdrag/" <> ToString@n1 <> d1 <> ToString@n2 <> d2 <> ...

4

Carl has shown a way to write to the file what you want, but I would suggest to not even put the terms= into the file: you can read the expression and set it to a variable when reading it. It will also get rid of all kinds of complications that occure if you want to write partially evaluated expressions. This approach lets you choose the name of the variable ...

4

Since you requested performance I would avoid Import and DateList and use ReadList and AbsoluteTime. format = {Number, Character, Number, Character, Number, Number, Character, Number, Character, Number, Number, Word}; data = {AbsoluteTime[{#5, #3, #1, #6, #8, #10}], ##11} & @@@ ReadList["data.txt", format]; "data.txt" is of course your data ...

4

For large files using ReadList is the way to go. So if we assume the name of your file is "file.txt", then the following should be efficient: StringReplace[#, {"e" -> "*^", "j" -> "I"}] & /@ ReadList["file.txt", Word, WordSeparators -> {"(", ")", " "}, RecordLists -> True] // ToExpression

3

I would convert your file first to CSV format. Then, there are a number of options for importing CSV files. If Import is too slow, you can try something similar to what is described here or here. The method described in the former link may require custom code, while the first method of the two described in the latter link is quite general and should work ...

3

You can use set stream position which sets the current point in an open stream. Then you can use Read which reads one expression from an input stream, and returns the expression with byte, character, expression, Number, Real, String or Word options. Something like: Edit 1 with ReadList n = 4; str = OpenRead["data.txt"]; Skip[str, Record, n - 1] ...

3

I use this way of storing data extensively for what I do. I collected in my answer here http://mathematica.stackexchange.com/a/999/66 several ideas I developed over time around this type of structure. The Keys function based on what dreeves once submitted on StackOverflow ( http://stackoverflow.com/a/154704/884752 ) is what makes this structure practical as ...

3

In version 7 halirutan's export method does not produce a file that is recognized by Import. However, one can write: Export["matrix2.h5.gz", datapourrie, {"GZIP", "HDF5"}] And then: d2 = Import["matrix2.h5.gz", {"Datasets", "/Dataset1"}]; datapourrie == d2 True

3

As @R.M. said in the comment, the function Put places the file into the directory returned by Directory[]. This location can be changed by using the function SetDirectory. I do not know of a way to directly write the expression terms=OutputFromOtherNotebook into Terms.m using Put, but I do have a method using the function Write. The following code writes ...

3

Depending on your needs it may be helpful to know that a significant amount of time is spent converting a Graphics expression to output Box forms. Example: graphics = Import["http://exampledata.wolfram.com/usamap.zip", "Graphics"]; Timing[cell = Cell[BoxData@ToBoxes@graphics, "Output"];] {1.466, Null} It is now significantly faster to print this ...

3

According to the comments you have several options: Use Import and Export to work with Excel files (noted: this is not straightforward with problems mentioned in the comments). Use this answer to manipulate the Excel file like it is done in VB. Using Leonid's SQL DSL as a way to get database like syntax while keeping all the data in Mathematica lists. I ...

Only top voted, non community-wiki answers of a minimum length are eligible