Mathematica Stack Exchange is a question and answer site for users of Mathematica. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

having here several files in ASCII format

I want to import, while removing headers and footers and then join the several lists.

strm = OpenRead[".omf"]
Skip[strm, Record, 38, NullRecords  -> True];
mydata = ReadList[strm, Number, 7500*3, RecordLists -> True]

This correctly removes headers and footers, so I don't know to combine this to a

alldata = Map[Import[ ], FileNames[]]

like command to import all files in the directory.

data = Import[#, "Table", HeaderLines -> 38, NumData ->  7500*3, 
AllNumeric -> False, TrailerLines -> 2] & /@ FileNames[]

This works for importing all files, but the footers (or trailers?) are not removed, output:

{{ 755205.327 391367.272 126872.268},{ 638270.884 407054.202 408065.135},{ 621838.43 593464.262 -26779.3974},{ 792199.887 301929.715 -144422.249},{ 827047.904 221093.933 -81909.9342},<<7493>>,{ 638270.884 407054.202 408065.135},
{ 755205.327 391367.272 126872.268},{# End: Data Text},{# End: Segment}}

HeaderLines isn't even mentioned in the Math7.0 Doc. In principle I only need the 3 column in every single .omf file

column3 = mydata[[All, 3]];

and those columns from every single file should than be arranged to a columnlenght X filenumber matrix list. Is it possible to apply this already when doing Import or OpenRead to save memory as this is just a example and the real files will be in the 2 digit GB range? Thanks for your tips

share|improve this question
up vote 3 down vote accepted

You can wrap your Skip and Readlist in a Module and make it a function:

importfile[name_] := Module[{strm, mydata},
  strm = OpenRead[name];
  Skip[strm, Record, 38, NullRecords -> True];
  mydata = ReadList[strm, Number, 7500*3, RecordLists -> True];


This will also likely be faster then import.

Alternatively, if you want to use Import, or in the case that your files are different lengths, you can import and then use Drop to remove the last 2 elements.

data = Drop[
    Import[#, "Table", HeaderLines -> 38, NumData -> 7500*3, 
     AllNumeric -> False, TrailerLines -> 2], -2] & /@ FileNames[]

If you want only the third column of very large files see: Reading periodic elements from a large file

share|improve this answer

A simplistic,if slightly inefficient, solution might be:

data = Import[#,"Table"][[39;;-3]]&/@Filenames[];

Other options might involve, 'Cases', 'Choices', Drop, Take.

For example:

data = Drop[Drop[Import[#,"Table"],38],-2]&/@Filenames[];

If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field from your files ( assumes whitespace separated data, other delimiters can be specified ).

data= ReadList["!head --lines=-2 " <>#<> " | tail --lines=+39 | cut -f 3 "]& /@ FileNames[]
share|improve this answer
Thanks, I posted a follow up question to this… – James Last Apr 29 '13 at 12:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.