I have a data file contains several lines of numbers that separated by "," and I cannot load it correctly with ReadList and Import.

Here is a example:

with data contains "," and "\n", simple ReadList doesn't work.

ReadList[StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"], Real]
(* {1099.5, $Failed} *)  If we use separators, it works, but if we group the types, it doesn't work. ReadList[StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"], Real, RecordSeparators -> {",", "\n"}] (* {1099.5, 0., -166.79, 1098.4, 0., -166.79} *) ReadList[StringToStream[ "1099.5,0,-166.79\n1098.4,0,-166.79\n"], {Real, Real, Real}, RecordSeparators -> {",", "\n"}] (* {$Failed} *)


And Import cannot import correctly as it treats each line as a string.

Import[StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"], "Table"]
StringQ /@
Flatten@Import[
StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"], "Table"]
(*
{{"1099.5,0,-166.79"}, {"1098.4,0,-166.79"}}
{True, True}

*)


So why they are not working in these cases? And How to make ReadList and Import to work without using Partition etc. after the import?

Version: 9.0 on Mac.

-

## closed as off-topic by m_goldberg, Ajasja, Artes, belisarius, RunnyKineNov 19 '13 at 14:38

This question appears to be off-topic. The users who voted to close gave this specific reason:

• "This question arises due to a simple mistake such as a trivial syntax error, incorrect capitalization, spelling mistake, or other typographical error and is unlikely to help any future visitors, or else it is easily found in the documentation." – m_goldberg, Ajasja, Artes, belisarius, RunnyKine
If this question can be reworded to fit the rules in the help center, please edit the question.

Could you give the reason why this question should be closed? I think this is a practical problem when I have a large data set to import. –  xslittlegrass Nov 19 '13 at 0:45
If I had to guess, it would be because you have comma separated values (CSV) and you're not using the direct option — "CSV" to do that. ReadList needs to have Record instead of Real. I think the latter is not very well known, but your question's main focus seems to be on performance with large files. You should focus on that instead. In any case, that would probably make this a dupe of the post that RunnyKine linked to... –  The Toad Nov 19 '13 at 1:46

Have you tried using "CSV" instead of "Table". Seems to work for me.

Import[StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"], "CSV"]

{{1099.5, 0, -166.79}, {1098.4, 0, -166.79}}


### Update

The problem with ReadList turned out to rather hard to track down. It seems that only white space can be used to delimited fields within records when ReadList is used. That is, the following will work.

ReadList[StringToStream["1099.5\t0\t-166.79\n1098.4\t0\t-166.79\n"],
{Number, Number, Number}]


{{1099.5, 0, -166.79}, {1098.4, 0, -166.79}}

and so will numerous variations of which I give a few examples.

ReadList[StringToStream["1099.5 0 -166.79\n1098.4 0 -166.79\n"],
{Number, Number, Number}]

Number, RecordLists -> True]

Number, RecordLists -> True]


Putting "," into the RecordSeparators list doesn't help because that makes every separate number a record, but you want three numbers in a record. Unfortunately, ReadList doesn't support a FieldSeparator option, which is what you really need,

-
Thanks. Do you know why ReadList doesn't work? I have a large data set and Import is quiet slow. –  xslittlegrass Nov 19 '13 at 0:49

Try this:

ReadList[StringToStream["1099.5,0,-166.79\n1098.4,0,-166.79\n"],
Table[Record, {3}], RecordSeparators -> {",", "\n"}] // ToExpression


{{1099.5, 0, -166.79}, {1098.4, 0, -166.79}}

-
Thanks, this work, but ToExpression seem drains the performance on large data files. –  xslittlegrass Nov 19 '13 at 0:48
@xslittlegrass No matter what the performance hit is (And I don't think it's much), it will still be much faster than Import. In terms of ReadList this is the best you can do. Also if you want to deal with large CSV files look here for my answer: mathematica.stackexchange.com/questions/35371/… –  RunnyKine Nov 19 '13 at 0:56