3
$\begingroup$

How are you?

I'm a chemist with almost no experience with Mathematica, but I'm thinking of using it as a tool for my Ph.D in Computational Chemistry. My Quantum Chemistry software (ORCA) generates a lot of output that is stored in text files. I know I can import text files to Mathematica (I know how to import such files) and that is it very powerful in manipulating texts. I want to import my outputs and, let's, say, to parse only the content between two strings. Here is an example or my output:

--------------
SCF ITERATIONS
--------------
ITER       Energy         Delta-E        Max-DP      RMS-DP      [F,P]     Damp
               ***  Starting incremental Fock matrix formation  ***
  0  -8693.9185205626   0.000000000000 0.02365877  0.00022580  0.2453968 0.8500
  1  -8694.4485310565  -0.530010493916 0.02170668  0.00021389  0.1786600 0.8500
                               ***Turning on DIIS***
  2  -8694.7699232947  -0.321392238231 0.01563958  0.00015675  0.1186396 0.8500
  3  -8694.9659372250  -0.196013930265 0.00723160  0.00008969  0.0765966 0.8500
  4  -8695.1123473361  -0.146410111134 0.00420004  0.00005953  0.0579101 0.8500
  5  -8695.2253413821  -0.112994046020 0.00296590  0.00004350  0.0478892 0.8500
  6  -8695.3172110457  -0.091869663593 0.00244669  0.00003472  0.0405933 0.8500
  7  -8695.3936024797  -0.076391433955 0.00228951  0.00002973  0.0345121 0.8500
  8  -8695.4579111293  -0.064308649595 0.01319047  0.00016820  0.0286770 0.0000
  9  -8695.8036488298  -0.345737700522 0.00420450  0.00002589  0.0109699 0.0000
 10  -8695.8031840358   0.000464794019 0.00295699  0.00000943  0.0232043 0.0000
 11  -8695.8041982923  -0.001014256490 0.00086373  0.00000677  0.0023797 0.0000
 12  -8695.8043427930  -0.000144500780 0.00052192  0.00000222  0.0004907 0.0000
 13  -8695.8043521928  -0.000009399784 0.00021592  0.00000091  0.0002833 0.0000
 14  -8695.8043542762  -0.000002083430 0.00014436  0.00000051  0.0002796 0.0000
 15  -8695.8043549153  -0.000000639004 0.00010908  0.00000041  0.0003521 0.0000
 16  -8695.8043553001  -0.000000384822 0.00011470  0.00000031  0.0002907 0.0000
 (...)
 60  -8695.8043562803   0.000000000015 0.00000455  0.00000001  0.0000020 0.0000
 61  -8695.8043562793   0.000000001011 0.00000442  0.00000001  0.0000020 0.0000
 62  -8695.8043562784   0.000000000917 0.00000429  0.00000001  0.0000019 0.0000
 63  -8695.8043562772   0.000000001197 0.00000417  0.00000001  0.0000019 0.0000
 64  -8695.8043562798  -0.000000002641 0.00000405  0.00000001  0.0000018 0.0000
                 **** Energy Check signals convergence ****

And I'm interested only in the numbers for the Iterations (most of all, it would be awesome if I could take sequences for each column).

Is there a way to get the numbers between "Starting incremental Fock matrix formation" and "Energy Check signals convergence" only?

Thank you.

EDIT: When looking to the whole output file, the number of lines usually change from one calculation to another, so, skipping a fixed number of first lines probably will not work.

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
4
$\begingroup$ 
file = "--------------
SCF ITERATIONS
--------------
ITER       Energy         Delta-E        Max-DP      RMS-DP      [F,P]     Damp
               ***  Starting incremental Fock matrix formation  ***
  0  -8693.9185205626   0.000000000000 0.02365877  0.00022580  0.2453968 0.8500
  1  -8694.4485310565  -0.530010493916 0.02170668  0.00021389  0.1786600 0.8500
                               ***Turning on DIIS***
  2  -8694.7699232947  -0.321392238231 0.01563958  0.00015675  0.1186396 0.8500
  3  -8694.9659372250  -0.196013930265 0.00723160  0.00008969  0.0765966 0.8500
  4  -8695.1123473361  -0.146410111134 0.00420004  0.00005953  0.0579101 0.8500
  5  -8695.2253413821  -0.112994046020 0.00296590  0.00004350  0.0478892 0.8500
  6  -8695.3172110457  -0.091869663593 0.00244669  0.00003472  0.0405933 0.8500
                 **** Energy Check signals convergence ****                    ";

start = StringPosition[file, "***Turning on DIIS***" ~~ " " ... ~~ "\n"][[1, 2]] + 1;
end = StringPosition[file, 
     "\n" ~~ " " ... ~~ "**** Energy Check signals convergence ****"][[1, 1]] - 1;
table = ImportString[StringTake[file, {start, end}], "Table"]

{{2, -8694.7699232947`, -0.321392238231`, 0.01563958`, 0.00015675`, 0.1186396`, 0.85`}, 
 {3, -8694.965937225`, -0.196013930265`, 0.0072316`, 0.00008969`, 0.0765966`, 0.85`},
 {4, -8695.1123473361`, -0.146410111134`, 0.00420004`, 0.00005953`, 0.0579101`, 0.85`},
 {5, -8695.2253413821`, -0.11299404602`, 0.0029659`, 0.0000435`, 0.0478892`, 0.85`},
 {6, -8695.3172110457`, -0.091869663593`, 0.00244669`, 0.00003472`, 0.0405933`, 0.85`}}

If your file contains some heads/comments between start and end, you can filter them out if you know the structure of your file, for example as Michael E2 suggests:

table2 = Cases[table, {_Integer, Repeated[_Real, {6}]}]

Another approach:

table2 = DeleteCases[table, {a_String, ___, a_} /; StringMatchQ[a, "*" ..]]
Improve this answer
$\endgroup$
4
  • $\begingroup$ Dear @Alexey, thank you, that looks like a very elegant solution and it does work for a snip of text but, unfortunately, when importing the whole output file I'm getting a series of errors: StringPosition::strse: String or list of strings expected at position 1 in StringPosition[{{},{ *****************},{ * O R C A *},<<45>>,{ Garnet Chan, S. Sharma, R. Olivares : DMRG},{ Ulf Ekstrom : XCFun DFT Library},<<1581835>>},***Turning on DIIS***~~ ...~~ ]. $\endgroup$
    – HCSthe2nd
    Commented Mar 13, 2017 at 3:05
  • $\begingroup$ @HenriqueJunior Use file = Import["myFile.txt", "Text"] for importing the file. $\endgroup$
    – Alexey Popkov
    Commented Mar 13, 2017 at 3:08
  • $\begingroup$ thanks! That did get rid of the error messages, but I'm getting some text inside my table. For example: Turning on DIIS and *** Restarting incremental Fock matrix formation *** appearing every 25 lines. Is there a way to get only numeric values? $\endgroup$
    – HCSthe2nd
    Commented Mar 13, 2017 at 3:26
  • $\begingroup$ @HenriqueJunior I updated the answer. $\endgroup$
    – Alexey Popkov
    Commented Mar 13, 2017 at 3:44
3
$\begingroup$

Using user9444 method to read text files with header. 

SetDirectory[NotebookDirectory[]]
data=Cases[Import["t.txt","Table"],{_?NumberQ,___}];
data

Mathematica graphics

MatrixForm[data]

Mathematica graphics

See skip-header-lines-on-import for other examples of how this can be done.

Matlab is a little easier in this sort of thing. It has textscan() function, which one can tell it how many lines to skip (for headers). I do not know why Mathematica does not have such functionality. The above method can fail, if header line happens to start with number as well. But for your file, it works.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ For skipping lines there are always things like Drop[Import[..], 5]. And one could make the pattern specific to the task; e.g. Cases[input, {_Integer, Repeated[_Real, {6}]}], provided you know the output format. $\endgroup$
    – Michael E2
    Commented Mar 13, 2017 at 2:42
  • $\begingroup$ @MichaelE2 good point about the Drop[Import[..], 5] command. That should work as well. $\endgroup$
    – Nasser
    Commented Mar 13, 2017 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.