# Data of badly formatted file

Similar as in this question, I have to read in data from a file (exampleFile.txt) which is not so well formatted. Unfortunately, I cannot find a solution by myself. Since it is a bit complicated to describe in which part of the data I am interested in at the moment, you can see in the screenshot below where I have marked in orange the data of interest.

Here is the code I got so far:

originalData=Import[file,"Table"];(*Read in of data*)
lengthOriginalData=Length[originalData];
mainData=originalData[[38;;lengthOriginalData-4]];(*take bulk of data*)
mainData1=Position[mainData,"Ion"];(*Search for position all data of interest have in common*)
positionToDelte=Table[{2*i},{i,1,Length[mainData1]/2}];(*there are unnecessary positions*)
mainData2=Delete[mainData1,positionToDelte];(*delete unnecessary positions*)
mainData3=mainData[[mainData2[[All,1]]+3]];(*take the data of interest*)
emptyLine=Position   [mainData3,"======================================================================================================"];(*there are still unnecessary lines*)
emptyLine2=Table[{emptyLine[[i,1]]},{i,1,Length[emptyLine]}];
dataOfInterest=Delete[mainData3,emptyLine2];(*delete the unnecessary lines*)
arrayOfInterest=dataOfInterest[[All,1]];(*this is the array in which the data of interest are*)
arrayOfInterest1=StringReplace[arrayOfInterest,"³"->","];(*a try to get the data of interest out of this array*)


As you can test, I get almost all I want (perhaps not with the nicest methods). Actually, all I would need is that I can take in my arrayOfInterest1 the right elements but somehow this does not work (I think the reason is that this is a string, but I need numbers).

-

Maybe you could do something like this:

First the data is imported as a list of strings.

import = Import["exampleFile.txt", "List"];


pos is the position of all lines containing the substring "Ion". The data of interest is located 3 lines below the lines containing "Ion". Therefore we omit the last four lines of import when searching.

pos = Flatten[Position[import[[;; -4]], a_String /; StringMatchQ[a, "*Ion*"], 1]];


Next we split the strings using the character "³" as the delimiter, and we transform the strings into numbers by replacing "E" in each substring with "*10^" and applying ToExpression.

strings = StringTrim /@ StringSplit[import[[pos + 3]], "³"];
data = Cases[strings, a_ /; Length[a] >= 5 :> ToExpression[
StringReplace[a[[3 ;; 5]], "E" -> "*10^"]]];


The first few entries for data are then

data[[;; 5]]

{{7618.6, -0.3894, 0.2726}, {3353.6, -0.0124, 0.03758},
{15268., 64.08, -8.825}, {13317., 38., 16.98}, {5654.9, 0.08338, 0.01426}}


which seems to match the screenshot.

-
WOW, Thank you so much Heike! That is a very smart solution! Thanks to @Sjoerd C. de Vries for edition my question and uploading my screenshot! –  partial81 Feb 27 '12 at 20:37

Time for regular expressions to save the day once again!

line = "³00001³96.15E+01³76186.E-01³-3894.E-04³ 2726.E-04³\
0005.05³ Si ³52049.E-04³000000001.000³      ³      ³"

(* Matches one number in the horrible log file: *)
regex = RegularExpression[
"(\\d+(\\.\\d*(e[+-]\\d+)?)?)"
] -> "\$1";
matches = StringCases[line, regex, IgnoreCase -> True];
(* Fix the exponential notation: *)
matches = StringReplace[
matches,
RegularExpression["e"] -> "*10^",
IgnoreCase -> True];
matches = ToExpression /@ matches

{1, 961.5, 7618.6, 0.3894, 0.2726, 5.05, 5.2049, 1.}


Time to beautify it a bit,

{{"Ion", "Energy", "Depth", "Lateral Y", "Lateral Z", "SE", "Recoil E", "Target"}, matches};
Transpose@% // Grid[#, Dividers -> All] &


-
Thank you! You showed very nice tricks with which I can display and use other data of this “horrible log file” :-) or data of similar horrible files. –  partial81 Feb 28 '12 at 9:09

Here is another option that should be pretty fast:

InternalStringToDouble /@ StringSplit[#, "\.b3"][[3 ;; 5]] & /@
Rest @ FindList["exampleFile.txt", "\n\.b3"]


Rest` is needed to skip one line in the header.

-