Skip to main content
added 315 characters in body
Source Link
Rolf Mertig
  • 17.4k
  • 1
  • 46
  • 78

A simple Mathematica-only solution is:

    CountLines[file_String /; FileExistsQ[file]] := 
    Module[{counter = 0, str = OpenRead@file},
           While[ Read[str, Record, NullRecords -> True] =!= EndOfFile, 
                  counter++
           ]; 
           Close[str];
           counter];

which is quite slow of course, so 123 MB (1978142 lines) needs 20 seconds in Mathematica 9 and 13 seconds in Mathematica 8. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

Another fast method on Windows is to use Gnu coreutils and then

AbsoluteTiming[
 ReadList["!\"C:\\Program Files (x86)\\GnuWin32\\bin\\wc\" -l < " <> 
    "I:\\allfiles.txt", Number] // First]

returns

{0.154009, 1978142}

A simple Mathematica-only solution is:

    CountLines[file_String /; FileExistsQ[file]] := 
    Module[{counter = 0, str = OpenRead@file},
           While[ Read[str, Record, NullRecords -> True] =!= EndOfFile, 
                  counter++
           ]; 
           Close[str];
           counter];

which is quite slow of course, so 123 MB (1978142 lines) needs 20 seconds in Mathematica 9 and 13 seconds in Mathematica 8. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

A simple Mathematica-only solution is:

    CountLines[file_String /; FileExistsQ[file]] := 
    Module[{counter = 0, str = OpenRead@file},
           While[ Read[str, Record, NullRecords -> True] =!= EndOfFile, 
                  counter++
           ]; 
           Close[str];
           counter];

which is quite slow of course, so 123 MB (1978142 lines) needs 20 seconds in Mathematica 9 and 13 seconds in Mathematica 8. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

Another fast method on Windows is to use Gnu coreutils and then

AbsoluteTiming[
 ReadList["!\"C:\\Program Files (x86)\\GnuWin32\\bin\\wc\" -l < " <> 
    "I:\\allfiles.txt", Number] // First]

returns

{0.154009, 1978142}
added 33 characters in body
Source Link
Rolf Mertig
  • 17.4k
  • 1
  • 46
  • 78

A simple Mathematica-only solution is:

    CountLines[file_String /; FileExistsQ[file]] := 
    Module[{counter = 0, str = OpenRead@file},
           While[ Read[str, Record]Record, NullRecords -> True] =!= EndOfFile, counter++];
                  counter++
           ]; 
           Close[str];
           counter];

which is quite slow of course, so 123 MB (18232821978142 lines) need 16needs 20 seconds on my laptopin Mathematica 9 and 13 seconds in Mathematica 8. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

A simple Mathematica-only solution is:

  CountLines[file_String /; FileExistsQ[file]] := 
  Module[{counter = 0, str = OpenRead@file},
   While[ Read[str, Record] =!= EndOfFile, counter++]; Close[str];
   counter];

which is quite slow of course, so 123 MB (1823282 lines) need 16 seconds on my laptop. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

A simple Mathematica-only solution is:

    CountLines[file_String /; FileExistsQ[file]] := 
    Module[{counter = 0, str = OpenRead@file},
           While[ Read[str, Record, NullRecords -> True] =!= EndOfFile, 
                  counter++
           ]; 
           Close[str];
           counter];

which is quite slow of course, so 123 MB (1978142 lines) needs 20 seconds in Mathematica 9 and 13 seconds in Mathematica 8. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.

Source Link
Rolf Mertig
  • 17.4k
  • 1
  • 46
  • 78

A simple Mathematica-only solution is:

  CountLines[file_String /; FileExistsQ[file]] := 
  Module[{counter = 0, str = OpenRead@file},
   While[ Read[str, Record] =!= EndOfFile, counter++]; Close[str];
   counter];

which is quite slow of course, so 123 MB (1823282 lines) need 16 seconds on my laptop. However, it only used 45 MB RAM at most (MaxMemoryUsed[]), so I guess you can easily count GB files.

I could not get Leonid's code to work on my Windows machine quickly, but if you can, that is of course better. Somehow Link technologies (JLink, NETLink,MathLink) will nearly always beat pure Mathematica. Unfortunately.