# Reading an ASCII file mixing text and numerics in Mathematica

We have a sequence of large data files (LAMMPS dump files) which begin like this:

ITEM: TIMESTEP
400000
ITEM: NUMBER OF ATOMS
22518
ITEM: BOX BOUNDS pp pp pp
0 527.757
0 532.678
-3.79681 3.79681
ITEM: ATOMS id x y z c_eng
22248 1.57153 2.05797 0 -0.700402
8 4.8285 3.9563 0 -0.683748
22250 5.24995 0.416325 0 -0.559546


The file continues for 22515 lines.

In the past we would delete the preamble lines through ITEM: ATOMS... and use ReadList to harvest the data. Now the file set is too large and we need to automate reading the file, header and all. That is, point to a data file and return numeric values for TIMESTEP, NUMBER OF ATOMS, BOX BOUNDS, and an array {id, x, y, z, energy}. What is the best use of Mathematica for such a case?

• These look like dump files generated by LAMMPS, the molecular dynamics simulator. Is that correct? I just wanted to leave that here in case somebody else searched the site for a similar answer in the future. – MarcoB Oct 13 '15 at 22:30
• @MarcoB: Good eye, good idea. Yes, a LAMPS output. – dantopa Oct 14 '15 at 16:12
• I just tried and think that LAMMPS in a comment is not good enough to find that question when searching for LAMMPS. Why not add it to the title of body text? I don't know if adding a tag is justified, though... – Albert Retey Oct 15 '15 at 10:13

Putting your data in a string stream instead of a real file for this demonstration. With a real file open it using OpenRead.

file = StringToStream[
"ITEM: TIMESTEP
400000
ITEM: NUMBER OF ATOMS
22518
ITEM: BOX BOUNDS pp pp pp
0 527.757
0 532.678
-3.79681 3.79681
ITEM: ATOMS id x y z c_eng
22248 1.57153 2.05797 0 -0.700402
8 4.8285 3.9563 0 -0.683748
22250 5.24995 0.416325 0 -0.559546 "];


Now get the parts of the header you need, skipping over the lines you don't need:

Skip[file, String, 1];
Skip[file, String, 1];
Skip[file, String, 1];
boxBounds = ReadList[file, Table[Number, {6}], 1];
Skip[file, String, 1];
Close[file];


Showing that we got the data:

{timeStep, numAtoms, boxBounds, data} // Column


Here is what I came up with. I am using Rule rather than Set so the variable names aren't defined as such. But you could definitely adapt your code to do that. I am assuming that the ITEM: label is present in all your output files.

bits = StringSplit[StringSplit[Import["test.txt"], "ITEM:"], "\n"]

(* {{" TIMESTEP", "400000"}, {" NUMBER OF ATOMS",
"22518"}, {" BOX BOUNDS pp pp pp", "0 527.757", "0 532.678",
"-3.79681 3.79681"}, {" ATOMS id x y z c_eng ",
"22248 1.57153 2.05797 0 -0.700402 ",
"8 4.8285 3.9563 0 -0.683748 ",
"22250 5.24995 0.416325 0 -0.559546 "}} *)

First[#] -> Map[ToExpression, StringSplit[Rest[#], Whitespace], {2}] & /@ bits

{" TIMESTEP" -> {{400000}}, " NUMBER OF ATOMS" -> {{22518}},
" BOX BOUNDS pp pp pp" -> {{0, 527.757}, {0, 532.678}, {-3.79681, 3.79681}},
" ATOMS id x y z c_eng " -> {{22248, 1.57153, 2.05797,
0, -0.700402}, {8, 4.8285, 3.9563, 0, -0.683748}, {22250, 5.24995,
0.416325, 0, -0.559546}}}


Further finessing could include using StringTrim to clean up the labels and using Flatten to make the first two values into a simple number rather than a nested list.