21
$\begingroup$

When importing a data file what are the comment symbols for Mathematica? That is, given a file like this

blabla
bulbul

1 2 6 54 7 ...
..

what symbol do I have to put in front of header lines so Mathematica skips them and starts reading at the line 1 2 6 54 7 .... I tried #, which works in gnuplot, but that did not work.

I know that I could just tell Mathematica to skip the lines, but as I can control the file output, it would be nicer to use some kind of a tag.

Improve this question
$\endgroup$
4
  • 7
    $\begingroup$ The simplest way is to Import[something][[n+1;;]] where n is the number of "commented" line you have. $\endgroup$
    – Öskå
    Commented Jun 13, 2014 at 12:18
  • $\begingroup$ possible duplicate of ReadList with header and foot lines $\endgroup$
    – rcollyer
    Commented Jun 13, 2014 at 12:40
  • 1
    $\begingroup$ At first glance, I thought this was a duplicate. It is not. The difference is that the number of header lines may be different each time, so it requires processing. $\endgroup$
    – rcollyer
    Commented Jun 13, 2014 at 12:41
  • $\begingroup$ @Öskå Are you able to answer this question here? I am not sure whether it is related: I have header with four lines where unique identifier is specified by two lines. $\endgroup$
    – hhh
    Commented Nov 11, 2014 at 3:17

4 Answers 4

Reset to default
18
$\begingroup$

Here is an approach that handles interspersed comments in addition to "headers"

 FilePrint["test.txt"]

#comment
#comment
#comment
1 2 3
#c2
4 5 6
7 8 9

 ImportString[
     StringReplace[Import["test.txt", "Text"], 
         StartOfLine ~~ "#" ~~ Shortest[___] ~~ EndOfLine ~~ "\n" -> ""], "Table"]

{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}

of course you can invent whatever convention you want or even a mix, eg.. 

 `StartOfLine ~~ {"#", "!", "%"} ~~ ...`

another variant:

 ImportString[StringJoin@Riffle[
      Select[StringSplit[Import["test.txt", "Text"], "\n"],
            StringTake[#, 1] != "#" &], "\n"], "Table"]

Even handle end-of-line comments:

#comment
1 2 3
#c2
4 5 6  #note 1
7 8 9  

 ImportString[StringReplace[Import["test.txt", "Text"], {
      StartOfLine ~~ "#" ~~ Shortest[___] ~~ EndOfLine ~~ "\n" -> "",
                     "#" ~~ Shortest[___] ~~ EndOfLine -> ""
                                       }], "Table"]

{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}

Tested on Windows by the way -- this might need some tweaking to handle different line endings on other systems

Improve this answer
$\endgroup$
4
  • $\begingroup$ Strictly speaking your last example doesn't handle in-line comments, but only handles comments that are terminated by a line ending. In-line comments need a beginning and an end delimiter. $\endgroup$
    – m_goldberg
    Commented Jun 13, 2014 at 15:48
  • $\begingroup$ ok.. not sure what to call that but I'll fix.. $\endgroup$
    – george2079
    Commented Jun 13, 2014 at 16:45
  • $\begingroup$ @george2079, does your apprach require ImportString or, in any case, can it be adapted to import CSV files, skipping blocks of rows (of varying length) consisting entirely of NullWords? $\endgroup$
    – alancalvitti
    Commented Nov 11, 2014 at 2:31
  • $\begingroup$ Are you able to answer this question here? I am not sure whether it is related: I have header with four lines where unique identifier is specified by two lines. $\endgroup$
    – hhh
    Commented Nov 11, 2014 at 3:16
19
$\begingroup$

I like to just import it and then filter it afterwards.

data = Cases[Import["file", "Table"], {_?NumberQ, ___}];

which will contain only those lines that start with a number.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I find this far more readable than george2079, upvote it! +1. Can someone explain why the other answer has got so many upvotes and this only 2? $\endgroup$
    – hhh
    Commented Nov 11, 2014 at 3:58
17
$\begingroup$

The Import command supports an option to ignore header lines. In many cases this is the easiest solution. For example:

dataStats = Import["C:/data/stats.csv", "CSV", HeaderLines -> 4];
Improve this answer
$\endgroup$
2
  • $\begingroup$ This does not work for white-space separated numbers, right? $\endgroup$
    – LeafGlowPath
    Commented Apr 12, 2019 at 14:08
  • $\begingroup$ Where is this Import option documented? $\endgroup$
    – Alan
    Commented Dec 4, 2019 at 16:11
5
$\begingroup$

Here's one possibility. I made a CSV file that imports looking like this:

data = Import["temp.csv"]
(* {{"header 1", ""}, {"header 2", ""}, {"#", ""}, {1, 1}, {2, 4}, {3, 9}, {4, 16}} *)

Search for your flag (I used #) using Position and then select all rows following that search

data[[Position[data, "#"][[1, 1]] + 1 ;;]]
(* {{1, 1}, {2, 4}, {3, 9}, {4, 16}} *)
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.