0
$\begingroup$

While I can read a data file containing the one/multiple line(s) of

1.314 -4.32 6.44 342.3

using

teststr = OpenRead["test_formatted.dat"];
Read[teststr, Table[Number, 4]]

I'm having difficulty reading a file which is formatted as

(1.314,0) (-4.32,1) (6.44,2) (342.3,3)

using any modification of above commands.

I want to avoid either (i) making new copies of original files and then format them externally using vim/sed [which can be done] because there are too many such huge files and I will run into memory constraints, or (ii) modifying the original files themselves to look like the first.

Can it be done with minimal changes to my above 2 lines of code?

$\endgroup$
1
$\begingroup$

The following is not very nice, but works:

data = Import["file.txt", "Table"];
ToExpression@StringReplace[#, {"(" -> "{", ")" -> "}"}] & /@ First@data

The idea is to convert the string to an expression; for that I first changed the parentheses to braces using StringReplace.

Might need a bit of fiddling if you use it for other file structures than the one you provided.

$\endgroup$
0
$\begingroup$

Probably the most efficient approach is to specify explicit sequence of object types to read:

file = "(-1.314,0) (4.32,1) (6.44,2) (342.3,3)
(-1.314,0) (4.32,1) (6.44,2) (342.3,3)";
stream = StringToStream[file];
ReadList[stream, Flatten@Table[{Character, Number, Character, Number, Character, Character}, 4]]
{{"(", -1.314`, ",", 0, ")", " ", "(", 4.32`, ",", 1, ")", " ", "(", 6.44`, ",", 2, ")", 
  " ", "(", 342.3`, ",", 3, ")", "\n"},
 {"(", -1.314`, ",", 0, ")", " ", "(", 4.32`, ",", 1, ")", " ", "(", 6.44`, ",", 2, ")", 
  " ", "(", 342.3`, ",", 3, ")", EndOfFile}}

Note however that the above solution assumes that line separators in your file are \n (Unix-style), not \r\n (Windows-style). In the latter case the above solution should be modified as follows:

ReadList[stream, Flatten[
  Table[{Character, Number, Character, Number, Character, Character}, 4]~Join~{Character}]]

Another approach is to parse the file using string expressions:

Partition[StringCases[file, n : NumberString :> ToExpression[n]], 8]
{{-1.314, 0, 4.32, 1, 6.44, 2, 342.3, 3}, 
 {-1.314, 0, 4.32, 1, 6.44, 2, 342.3, 3}}

This solution will be slower than the ReadList-based method.

A bit faster (but dangerous) is to use undocumented Internal`StringToDouble instead of ToExpression:

Partition[StringCases[file, n : NumberString :> Internal`StringToDouble[n]], 8]
{{-1.314, 0., 4.32, 1., 6.44, 2., 342.3, 3.}, 
 {-1.314, 0., 4.32, 1., 6.44, 2., 342.3, 3.}}

If you run into memory constraints with this solution, you can process the file line-by-line:

file = "(-1.314,0) (4.32,1) (6.44,2) (342.3,3)
(-1.314,0) (4.32,1) (6.44,2) (342.3,3)";
stream = StringToStream[file];
Reap[While[(rec = Read[stream, Record]) =!= EndOfFile,
   Sow@StringCases[rec, n : NumberString :> Internal`StringToDouble[n]]]][[2, 1]]    
{{-1.314, 0., 4.32, 1., 6.44, 2., 342.3, 3.}, 
 {-1.314, 0., 4.32, 1., 6.44, 2., 342.3, 3.}}

Don't forget to close the stream after importing the data:

Close[stream];
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.