Is there a solution to skip comments in text files that are imported with
OpenRead? I know there are solutions for
ImportStringbut I cannot find any way to handle the streams.
Imagine I have the file:
#comment #comment 1 2 3 # another comment 4 5 6 7 8 9 1 3 4 # another comment
Also notice that there are whitespaces before comments and before data. Finally, the comments are effectively at random places such that no repetition pattern can be inferred a priori.
I was thinking of
Skipbut it seems to have only fixed types so I probably cannot use a pattern matching.
There is a similar solution here but it does not address the specific situation when a line starts with a specific character. At this moment I am thinking of either implementing the slow
Import or preprocessing with Python but I bet there must be a simple way with Mathematica.
A second problem whether I can use the stream to extract data right away (in the example above I would like to know the values in columns 2 and 3 if the value in the first column equals 1. I know how to do this with data from
Importbut I am not sure about how to do it with the stream. I found some code utilizing
EndOfFile but as everybody says do not use loops in Mathematica, what else is available?
Also, it would be great if the whole solution was parallelizable (I need to read large number of files, process them --- extract data --- then put the extracted data into a common list).