Skip to main content
replaced http://mathematica.stackexchange.com/ with https://mathematica.stackexchange.com/
Source Link
  • The countLines function is taken from this answer of Mr. Wizard.

  • The combination of ReadList with ImportString and Riffle is taken from an answer in the same post here by Leonid Shifrin.

  • The best part of the code, which is the fastest reading with ReadLine based on user specified types, is taken from this answer of Mr. Wizard. It would be great if someone analyze how exactly this blank, skipping characters work and whether I am using them appropriately here.

  • The countLines function is taken from this answer of Mr. Wizard.

  • The combination of ReadList with ImportString and Riffle is taken from an answer in the same post here by Leonid Shifrin.

  • The best part of the code, which is the fastest reading with ReadLine based on user specified types, is taken from this answer of Mr. Wizard. It would be great if someone analyze how exactly this blank, skipping characters work and whether I am using them appropriately here.

correct hyperlink to point to the apparently referenced post
Source Link
Mr.Wizard
  • 273.1k
  • 34
  • 595
  • 1.4k
  • The countLines function is taken from this answer of Mr. Wizard.

  • The combination of ReadList with ImportString and Riffle is taken from an answer in the same post here by Leonid Shifrin.

  • The best part of the code, which is the fastest reading with ReadLine based on user specified types, is taken from this answerfrom this answer of Mr. Wizard. It would be great if someone analyze how exactly this blank, skipping characters work and whether I am using them appropriately here.

  • The countLines function is taken from this answer of Mr. Wizard.

  • The combination of ReadList with ImportString and Riffle is taken from an answer in the same post here by Leonid Shifrin.

  • The best part of the code, which is the fastest reading with ReadLine based on user specified types, is taken from this answer of Mr. Wizard. It would be great if someone analyze how exactly this blank, skipping characters work and whether I am using them appropriately here.

  • The countLines function is taken from this answer of Mr. Wizard.

  • The combination of ReadList with ImportString and Riffle is taken from an answer in the same post here by Leonid Shifrin.

  • The best part of the code, which is the fastest reading with ReadLine based on user specified types, is taken from this answer of Mr. Wizard. It would be great if someone analyze how exactly this blank, skipping characters work and whether I am using them appropriately here.

added 127 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23

If you want to read only the header of the file then specify start->0, end->0

null[_String] := Null
countLines = Length@ReadList[#, null@String, NullRecords -> True] &;

ClearAll[readRows];
Options[readRows]={filetype->"CSV",start->0,end->-1, types->{}, import->True, find->"", header->True};
readRows[fname_String?FileExistsQ,OptionsPattern[]]:=Module[
{txtstrm, ftype,typesval,impmethod,
 startLine,endLine,line1,str, 
 hdr, hdrflag, body, bodyImport, blnk},

(* Blank function for skipping word separators, e.g. commas *)
blnk[]=blnk[Character];
blnk[_String]=Sequence[];

(* Get Option Values *)
startLine=OptionValue[start];endLine=OptionValue[end];
ftype=OptionValue[filetype];typesval=OptionValue[types];
impmethod=OptionValue[import];str=OptionValue[find];
hdrflag=OptionValue[header] ;

(* Open Stream *)
txtstrm=OpenRead[fname];

(* Process Header, i.e. first line *)
line1 = ReadList[txtstrm,Record,1][[1]];
Which[
 ftype=="CSV", hdr = StringSplit[line1,","],
 ftype=="TSV", hdr = StringSplit[line1,"\t"],
 True, Abort[] ];

(* If user has not specified any types for ReadLine, types\[Rule]{} *)
If[Length@typesval==0,typesval= Riffle[Table[Word,Length@hdr],blnk[]]];

(* If user has not specified the line to end reading *)
If[endLine==-1,endLine=countLines@fname];

(* If user specified to start reading the text stream from a specific line *)
If [startLine!=0,Skip[txtstrm,Record,startLine]];

(* If user specified a string to find in a file *)
If[str!="",Find[txtstrm,str]];

(* Check the import option for post-processing each row with ImportString *)
If[impmethod,
 body =ReadList[txtstrm,Record,endLine-startLine]; 
 bodyImport= ImportString[StringJoin[Riffle[body,"\n"]],ftype,CharacterEncoding->"UTF-8"],
(* else *)
  Which[
    ftype=="CSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n",","}],
    ftype=="TSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n","\t"}] ];    
]; (* End of If impmethod *)

(* Close the stream *)
Close[txtstrm];

(* Optionally add the header at the beginning, return the result *)
Which[
    startLine==0 && endLine==0, hdr,
    impmethod && hdrflag,   Prepend[bodyImport, hdr],
    impmethod,              bodyImport,
    hdrflag,                Prepend[body, hdr],
    True,                   body]

] (* End of readRows *)
null[_String] := Null
countLines = Length@ReadList[#, null@String, NullRecords -> True] &;

ClearAll[readRows];
Options[readRows]={filetype->"CSV",start->0,end->-1, types->{}, import->True, find->"", header->True};
readRows[fname_String?FileExistsQ,OptionsPattern[]]:=Module[
{txtstrm, ftype,typesval,impmethod,
 startLine,endLine,line1,str, 
 hdr, hdrflag, body, bodyImport, blnk},

(* Blank function for skipping word separators, e.g. commas *)
blnk[]=blnk[Character];
blnk[_String]=Sequence[];

(* Get Option Values *)
startLine=OptionValue[start];endLine=OptionValue[end];
ftype=OptionValue[filetype];typesval=OptionValue[types];
impmethod=OptionValue[import];str=OptionValue[find];
hdrflag=OptionValue[header] ;

(* Open Stream *)
txtstrm=OpenRead[fname];

(* Process Header, i.e. first line *)
line1 = ReadList[txtstrm,Record,1][[1]];
Which[
 ftype=="CSV", hdr = StringSplit[line1,","],
 ftype=="TSV", hdr = StringSplit[line1,"\t"],
 True, Abort[] ];

(* If user has not specified any types for ReadLine, types\[Rule]{} *)
If[Length@typesval==0,typesval= Riffle[Table[Word,Length@hdr],blnk[]]];

(* If user has not specified the line to end reading *)
If[endLine==-1,endLine=countLines@fname];

(* If user specified to start reading the text stream from a specific line *)
If [startLine!=0,Skip[txtstrm,Record,startLine]];

(* If user specified a string to find in a file *)
If[str!="",Find[txtstrm,str]];

(* Check the import option for post-processing each row with ImportString *)
If[impmethod,
 body =ReadList[txtstrm,Record,endLine-startLine]; 
 bodyImport= ImportString[StringJoin[Riffle[body,"\n"]],ftype,CharacterEncoding->"UTF-8"],
(* else *)
  Which[
    ftype=="CSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n",","}],
    ftype=="TSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n","\t"}] ];    
]; (* End of If impmethod *)

(* Close the stream *)
Close[txtstrm];

(* Optionally add the header at the beginning, return the result *)
Which[
    impmethod && hdrflag,   Prepend[bodyImport, hdr],
    impmethod,              bodyImport,
    hdrflag,                Prepend[body, hdr],
    True,                   body]

] (* End of readRows *)

If you want to read only the header of the file then specify start->0, end->0

null[_String] := Null
countLines = Length@ReadList[#, null@String, NullRecords -> True] &;

ClearAll[readRows];
Options[readRows]={filetype->"CSV",start->0,end->-1, types->{}, import->True, find->"", header->True};
readRows[fname_String?FileExistsQ,OptionsPattern[]]:=Module[
{txtstrm, ftype,typesval,impmethod,
 startLine,endLine,line1,str, 
 hdr, hdrflag, body, bodyImport, blnk},

(* Blank function for skipping word separators, e.g. commas *)
blnk[]=blnk[Character];
blnk[_String]=Sequence[];

(* Get Option Values *)
startLine=OptionValue[start];endLine=OptionValue[end];
ftype=OptionValue[filetype];typesval=OptionValue[types];
impmethod=OptionValue[import];str=OptionValue[find];
hdrflag=OptionValue[header] ;

(* Open Stream *)
txtstrm=OpenRead[fname];

(* Process Header, i.e. first line *)
line1 = ReadList[txtstrm,Record,1][[1]];
Which[
 ftype=="CSV", hdr = StringSplit[line1,","],
 ftype=="TSV", hdr = StringSplit[line1,"\t"],
 True, Abort[] ];

(* If user has not specified any types for ReadLine, types\[Rule]{} *)
If[Length@typesval==0,typesval= Riffle[Table[Word,Length@hdr],blnk[]]];

(* If user has not specified the line to end reading *)
If[endLine==-1,endLine=countLines@fname];

(* If user specified to start reading the text stream from a specific line *)
If [startLine!=0,Skip[txtstrm,Record,startLine]];

(* If user specified a string to find in a file *)
If[str!="",Find[txtstrm,str]];

(* Check the import option for post-processing each row with ImportString *)
If[impmethod,
 body =ReadList[txtstrm,Record,endLine-startLine]; 
 bodyImport= ImportString[StringJoin[Riffle[body,"\n"]],ftype,CharacterEncoding->"UTF-8"],
(* else *)
  Which[
    ftype=="CSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n",","}],
    ftype=="TSV", body =ReadList[txtstrm,typesval,endLine-startLine,WordSeparators->{"\n","\t"}] ];    
]; (* End of If impmethod *)

(* Close the stream *)
Close[txtstrm];

(* Optionally add the header at the beginning, return the result *)
Which[
    startLine==0 && endLine==0, hdr,
    impmethod && hdrflag,   Prepend[bodyImport, hdr],
    impmethod,              bodyImport,
    hdrflag,                Prepend[body, hdr],
    True,                   body]

] (* End of readRows *)
added 225 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 19 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 12 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 171 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 2 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 2 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 179 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 179 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 179 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 6 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 165 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 165 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 614 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 1518 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 1518 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 1518 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 1518 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 12 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
deleted 7 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
added 205 characters in body
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading
Source Link
Athanassios
  • 1.3k
  • 10
  • 23
Loading