7
$\begingroup$

I am trying to read from a stream of strings and numbers delimited by commas similar to the way I have read from a stream of strings and numbers delimited by spaces.


Example delimited by spaces (which works):

stream = StringToStream["Apple Jack 1 123.456\nOrange Jill 2 456.789\n"];
While[! EndOfFile  === (data = 
      Read[stream, { Word, Word, Number, Number }]),
 Print["Fruit:", data[[1]], " Name:", data[[2]],
      " Integer:", data[[3]], " Real:", data[[4]]];
]

Results:

Fruit:Apple Name:Jack Integer:1 Real:123.456
Fruit:Orange Name:Jill Integer:2 Real:456.789

Example delimited by commas (which does not work):

stream = StringToStream["Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];
While[! EndOfFile  === (data = 
      Read[stream, { Word, Word, Number, Number }]),
 Print["Fruit:", data[[1]], " Name:", data[[2]],
      " Integer:", data[[3]], " Real:", data[[4]]];
]

Results:

Fruit:Apple,Jack,1,123.456 Name:Orange,Jill,2,456.789 Integer:EndOfFile Real:EndOfFile

It appears that commas are not default separators.


I added WordSeparators -> "," as an option to Read. Note NumberSeparator is not valid option for Read.

stream = StringToStream["Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];
While[! EndOfFile  === (data = 
      Read[stream, { Word, Word, Number, Number }, {WordSeparators -> ","}]),
 Print["Fruit:", data[[1]], " Name:", data[[2]],
       " Integer:", data[[3]], " Real:", data[[4]]];
 ]

Results:

Read::readn: Invalid real number found when reading from StringToStream[Apple,Jack,1,123.456
Orange,Jill,2,456.789
]. >>

Fruit:Apple Name:Jack Integer:$Failed Real:EndOfFile

Read::readn: Invalid real number found when reading from StringToStream[Apple,Jack,1,123.456
Orange,Jill,2,456.789
]. >>

Fruit:1 Name:123.456 Integer:$Failed Real:EndOfFile
...

Well this works for words, but not for numbers.


I can always do this, but this seems a little overkill:

stream = StringToStream["Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];
While[! EndOfFile  === (record = Read[stream, Record]),
 data = MapAt[ToExpression, StringSplit[record, ","], {{3}, {4}}];
 Print["Fruit:", data[[1]], " Name:", data[[2]],
      " Integer:", data[[3]], " Real:", data[[4]]];
]

Results:

Fruit:Apple Name:Jack Integer:1 Real:123.456
Fruit:Orange Name:Jill Integer:2 Real:456.789

Is there something thing that I am missing with Read?

Post answer analysis ...

The check was awarded to the answer that best answered the above question as I am not one to change the rules of the game while it is being played. However the question was flawed by an assumption that I made that both the parsing and conversion in Read would be the most efficient.

If one is looking for the most efficient solution, the following are some results from answers presented (timings are in seconds for 10000 iterations):

        Implementation         |  Timing 
-------------------------------+---------
question - Read[] with spaces  | 0.220325
Matariki - wo/Sequence[]       | 0.250499
Matariki - w/Sequence[]        | 0.345603
Heike - Word, Word, Word, Word | 0.351137
Heike - Riffle                 | 0.380684
question - overkill            | 0.395058
$\endgroup$
6
  • $\begingroup$ I guess you could read all records as words and use ToExpression to turn them into numbers. $\endgroup$
    – Heike
    Feb 23, 2012 at 20:18
  • $\begingroup$ Have you tried instead of using Word, Word, Number, Number to use Word, Word, Number, Real? $\endgroup$
    – Matariki
    Feb 23, 2012 at 20:23
  • $\begingroup$ @Matariki - yeap same results as Word, Word, Number, Number. $\endgroup$
    – mmorris
    Feb 23, 2012 at 20:26
  • $\begingroup$ @Heike - Yeap ... your suggestion no longer requires splitting, but still requires the conversion. I was hopping someone knew of a super secret option to Read that would allow commas as separators. I guess I must be misusing yet another Mathematica function for domination of the world. ;) $\endgroup$
    – mmorris
    Feb 23, 2012 at 20:52
  • $\begingroup$ I'd contend that you're seeing a bug. The problem is the comma between the two number's. Simple testing using Word shows that the comma is not removed until the next token is pulled from the stream, and while Word will respect the WordSeparators, Number does not. It insists on whitespace only. It seems clear that WordSeparators is intended as a list of field separators, but it doesn't act that way. $\endgroup$
    – rcollyer
    Feb 24, 2012 at 2:41

3 Answers 3

6
$\begingroup$

This seems to work and doesn't require conversion

stream = StringToStream[
   "Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];

While[! EndOfFile === (data = 
      Read[stream, Riffle[{Word, Word, Number, Number}, Word],
        TokenWords -> {","}] /. {"," -> Sequence[]}), 
  Print["Fruit:", data[[1]], " Name:", data[[2]], " Integer:", 
    data[[3]], " Real:", data[[4]]];];

Close[stream];
$\endgroup$
2
  • $\begingroup$ Nice. No need for extra storage space in data. +1 $\endgroup$
    – Matariki
    Feb 23, 2012 at 20:49
  • $\begingroup$ Earns the check (unless that super secret Read option shows up) because it is a more general purpose solution. $\endgroup$
    – mmorris
    Feb 23, 2012 at 21:27
7
$\begingroup$

Try this:

stream = StringToStream[
   "Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];
While[! EndOfFile === (data = 
     Read[stream, {Word, Word, Character, Number, Character, Number}, 
      WordSeparators -> ","]), 
 Print["Fruit:", data[[1]], " Name:", data[[2]], " Integer:", 
   data[[4]], " Real:", data[[6]]];]

Having learnd something from Heike's answer I add /. {"," -> Sequence[]} which removes the commmas and there is no need to skip the them in data.

stream = StringToStream[
   "Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n"];
While[! EndOfFile === (data = 
     Read[stream, {Word, Word, Character, Number, Character, Number}, 
      WordSeparators -> ","] /. {"," -> Sequence[]}), 
 Print["Fruit:", data[[1]], " Name:", data[[2]], " Integer:", 
   data[[3]], " Real:", data[[4]]];]
$\endgroup$
0
2
$\begingroup$

Have you considered using ImportString?

ImportString["Apple,Jack,1,123.456\nOrange,Jill,2,456.789\n", "CSV"]
{{"Apple", "Jack", 1, 123.456}, {"Orange", "Jill", 2, 456.789}}
$\endgroup$
1
  • $\begingroup$ I have used Import and ImportString in the past, unfortunately in this particular instance I need to read from a stream. The use of stream = StringToStream["Apple Jack 1 123.456\nOrange Jill 2 456.789\n"]; is contrived so I could give a concise, but accurate example of my problem. $\endgroup$
    – mmorris
    Feb 24, 2012 at 3:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.