I'm trying to figure out how many rows/records my csv file has, which I have to stream because it's quite large.

Can someone tell me if my approach works and how I can get it to loop through the whole data. It works for the n rows I put in the code but since I don't know what number of the last row is I can't get it to work. Basically, I don't know how to finish the Do command properly where I put the ?????.

stream = OpenRead["C:\\Users\\Name\\Documents\\file.csv"];
list = {};
  record = Read[stream, Record];
  AppendTo[list, ToExpression @ First @ StringSplit[record, ","]], 
entryCount = Length[list]
  • $\begingroup$ Read will return EndOfFile if you attempt to read past the end of the stream. $\endgroup$ Commented Feb 26, 2019 at 1:04
  • $\begingroup$ This does not implement CSV (need to check delimiters because a new line might be in the data, or it might indicate the end of a row).. Take a look at this solution for now (mathematica.stackexchange.com/a/183652/38159), but I can tell you next version will make this much easier and use much less memory, so don't stick with these Read approaches past 11.3 $\endgroup$ Commented Feb 26, 2019 at 3:28

2 Answers 2


I have two main recommendations:

  1. Forget Do and use While to stop when Read returns EndOfFile.
  2. Write defensive code when dealing with the file system.

With those things in mind, the following code seems a reasonable solution to your problem.

readCSV[path_] :=
  Module[{strm, recd, data = {}},
      If[(strm = OpenRead[path]) === $Failed, Return[$Failed]];
      While[(recd = Read[strm, Record]) =!= EndOfFile,
        AppendTo[data, ToExpression["{" <> recd <> "}"]]];


path = FileNameJoin[{$HomeDirectory, "Desktop", "data.csv"}];
Export[path, Table[{i, i^2, i^3}, {i, 5}]];
{{1, 1, 1}, {2, 4, 8}, {3, 9, 27}, {4, 16, 64}, {5, 25, 125}}`
  • $\begingroup$ OP says "which I have to stream because it's quite large". How does this solution help with that? The entire file is read. $\endgroup$ Commented Feb 26, 2019 at 1:08
  • $\begingroup$ @RohitNamjoshi. It gives the OP a better code structure to work with for reading records in blocks. I didn't write the blocking part because the OP didn't give any information on how the handle the blocks. $\endgroup$
    – m_goldberg
    Commented Feb 26, 2019 at 1:20

What's wrong with

list = ReadList[stream, Records]
  • $\begingroup$ ReadList::readf: Records is not a valid format specification. $\endgroup$ Commented Feb 26, 2019 at 0:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.