# Reading CSV data from a stream [duplicate]

I'm trying to figure out how many rows/records my csv file has, which I have to stream because it's quite large.

Can someone tell me if my approach works and how I can get it to loop through the whole data. It works for the n rows I put in the code but since I don't know what number of the last row is I can't get it to work. Basically, I don't know how to finish the Do command properly where I put the ?????.

stream = OpenRead["C:\\Users\\Name\\Documents\\file.csv"];
list = {};
Do[
AppendTo[list, ToExpression @ First @ StringSplit[record, ","]],
{?????}];
entryCount = Length[list]

• Read will return EndOfFile if you attempt to read past the end of the stream. – Rohit Namjoshi Feb 26 '19 at 1:04
• This does not implement CSV (need to check delimiters because a new line might be in the data, or it might indicate the end of a row).. Take a look at this solution for now (mathematica.stackexchange.com/a/183652/38159), but I can tell you next version will make this much easier and use much less memory, so don't stick with these Read approaches past 11.3 – GenericAccountName Feb 26 '19 at 3:28

I have two main recommendations:

1. Forget Do and use While to stop when Read returns EndOfFile.
2. Write defensive code when dealing with the file system.

With those things in mind, the following code seems a reasonable solution to your problem.

readCSV[path_] :=
Module[{strm, recd, data = {}},
CheckAbort[
If[(strm = OpenRead[path]) === $$Failed, Return[$$Failed]];
While[(recd = Read[strm, Record]) =!= EndOfFile,
AppendTo[data, ToExpression["{" <> recd <> "}"]]];
Close[strm];
data,
Close[strm]]]


Test

path = FileNameJoin[{\$HomeDirectory, "Desktop", "data.csv"}];
Export[path, Table[{i, i^2, i^3}, {i, 5}]];

{{1, 1, 1}, {2, 4, 8}, {3, 9, 27}, {4, 16, 64}, {5, 25, 125}}

list = ReadList[stream, Records]

• ReadList::readf: Records is not a valid format specification.` – Rohit Namjoshi Feb 26 '19 at 0:57