2
$\begingroup$

I have a data output from a device in a form of a string. I need to transform it into such a form that I can work with it, such as plot columns with respect to one another and such. After a apply Characters to it it takes the form:

   lst={" ", " ", " ", "1", "6", ".", "4", "1", "0", "7", " ", " ", " ", " \
", " ", " ", " ", " ", " ", " ", " ", "4", ".", "4", "3", "1", "5", \
"e", "-", "0", "1", " ", " ", " ", " ", " ", " ", " ", "5", ".", "4", \
"0", "0", "7", "e", "+", "0", "2", " ", " ", " ", " ", " ", " ", "1", \
".", "9", "8", "3", "8", "e", "-", "0", "1", " ", " ", " ", " ", " ", \
" ", " ", "2", ".", "4", "1", "7", "7", "e", "+", "0", "2", " ", " ", \
" ", " ", " ", " ", " ", "7", ".", "4", "8", "4", "5", "e", "+", "0", \
"0", " ", " ", " ", " ", " ", " ", " ", "9", ".", "1", "2", "1", "5", \
"e", "+", "0", "3", "\\n", " ", " ", " ", " ", " ", " ", "1", "8", \
".", "6", "8", "2", "5", " ", " ", " ", " ", " ", " ", " ", " ", " ", \
" ", " ", "1", ".", "2", "2", "0", "1", "e", "+", "0", "0", " ", " ", \
" ", " ", " ", " ", " ", "1", ".", "3", "7", "1", "8", "e", "+", "0", \
"3", " ", " ", " ", " ", " ", " ", "3", ".", "3", "6", "3", "5", "e", \
"-", "0", "1", " ", " ", " ", " ", " ", " ", " ", "3", ".", "6", "0", \
"0", "7", "e", "+", "0", "2", " ", " ", " ", " ", " ", " ", " ", "1", \
".", "4", "4", "5", "1", "e", "+", "0", "1", " ", " ", " ", " ", " ", \
" ", " ", "1", ".", "5", "4", "7", "0", "e", "+", "0", "4", "\\n", " \
", " ", " ", " ", " ", " "}

This is a small part of the data. StringJoinyields 7 columns, which is how the data is organized

enter image description here

So, any idea of how to transform it into a normal list?

Later edit: to address the question of bbgodfrey: this is a part of the original output. I add it as an image:

enter image description here

After I transformed in into characters and cut away first 1360 ones containing the useless billhead, and then combine them back to the string we see 7 columns with numbers that I already posted above.

Improve this question
$\endgroup$
10
  • $\begingroup$ As a rough start, try something like SemanticImportString[StringJoin@StringDelete[lst, "\\n"]]. This will give you a Dataset from which you could get your data, and possibly Partition the result into a usable table. $\endgroup$
    – MarcoB
    Oct 26, 2015 at 18:52
  • $\begingroup$ You mean you want to convert the whole lot into the real-valued list {16.4107, 4.4315e-01, …}? $\endgroup$
    – Patrick Stevens
    Oct 26, 2015 at 18:52
  • $\begingroup$ What does the original output look like? If one giant string, can you post a part of it? $\endgroup$
    – bbgodfrey
    Oct 26, 2015 at 19:05
  • 1
    $\begingroup$ Looks like the kind of job that fits like a glove to Awk, Perl or similars $\endgroup$
    – Dr. belisarius
    Oct 26, 2015 at 19:25
  • 1
    $\begingroup$ @AlexeiBoulbitch If you're going to work a lot with semi-formatted text files you should try them. Awk is easier, Perl is more modern and powerful. Google them :) $\endgroup$
    – Dr. belisarius
    Oct 26, 2015 at 20:01

2 Answers 2

Reset to default
2
$\begingroup$

Mathematica has a very rich set of methods to deal with strings.

lst = {" ", " ", " ", "1", "6", ".", "4", "1", "0", "7", " ", " ", 
  " ", " ", " ", " ", " ", " ", " ", " ", " ", "4", ".", "4", "3", 
  "1", "5", "e", "-", "0", "1", " ", " ", " ", " ", " ", " ", " ", 
  "5", ".", "4", "0", "0", "7", "e", "+", "0", "2", " ", " ", " ", 
  " ", " ", " ", "1", ".", "9", "8", "3", "8", "e", "-", "0", "1", 
  " ", " ", " ", " ", " ", " ", " ", "2", ".", "4", "1", "7", "7", 
  "e", "+", "0", "2", " ", " ", " ", " ", " ", " ", " ", "7", ".", 
  "4", "8", "4", "5", "e", "+", "0", "0", " ", " ", " ", " ", " ", 
  " ", " ", "9", ".", "1", "2", "1", "5", "e", "+", "0", "3", "\\n", 
  " ", " ", " ", " ", " ", " ", "1", "8", ".", "6", "8", "2", "5", 
  " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "1", ".", 
  "2", "2", "0", "1", "e", "+", "0", "0", " ", " ", " ", " ", " ", 
  " ", " ", "1", ".", "3", "7", "1", "8", "e", "+", "0", "3", " ", 
  " ", " ", " ", " ", " ", "3", ".", "3", "6", "3", "5", "e", "-", 
  "0", "1", " ", " ", " ", " ", " ", " ", " ", "3", ".", "6", "0", 
  "0", "7", "e", "+", "0", "2", " ", " ", " ", " ", " ", " ", " ", 
  "1", ".", "4", "4", "5", "1", "e", "+", "0", "1", " ", " ", " ", 
  " ", " ", " ", " ", "1", ".", "5", "4", "7", "0", "e", "+", "0", 
  "4", "\\n", " ", " ", " ", " ", " ", " "}

For this answer it will be easier if the string is not broken into characters.

text = StringJoin[lst];

Step 1

Replace the letter "e" with the string "*10^".

text1 = StringReplace[text, "e" -> "*10^"]

Step 2

Break the complete string into a list of lines.

lines = StringSplit[text1, "\\n"]

For your actual data you would remove the first few lines until you get down to the column of numbers.

Step 3

This is the workhorse portion.

We will take the contiguous characters between the spaces and convert them to a number.

data = Map[
  StringCases[#, 
    WordBoundary ~~ x : Characters["01234567890-+.*^"] .. ~~ 
      WordBoundary :> ToExpression[x]] &,
  lines
  ]

The output is

{{16.4107, 0.44315, 540.07, 0.19838, 241.77, 7.4845, 
  9121.5}, {18.6825, 1.2201, 1371.8, 0.33635, 360.07, 14.451, 
  15470.}, {}}

Step 4

Observe that there is an empty list. The last step is to remove all empty lists.

data = DeleteCases[data, {}]

yielding the final result

{{16.4107, 0.44315, 540.07, 0.19838, 241.77, 7.4845, 
  9121.5}, {18.6825, 1.2201, 1371.8, 0.33635, 360.07, 14.451, 15470.}}
Improve this answer
$\endgroup$
2
  • $\begingroup$ Jack, this looks very promising, but by application to my txt file I get unexpected results on the step 3. The warning: ToExpression::sntx: Invalid syntax in or before "*10^". and instead of the removing the whole "cap" each word is replaced by $Failed. Besides, the figures are not splitted into sublists, as expected. The most simple would be to show you the initial txt file, but I do not know, how to do it. $\endgroup$
    – Alexei Boulbitch
    Oct 27, 2015 at 12:57
  • $\begingroup$ @AlexeiBoulbitch I looked at the text from your answer. For that text in Step 2 I had to use: lines = StringSplit[text1, EndOfLine]. The rest of it, Step 3 included worked fine. Make sure if you go to your most raw text you removed the lines that precede the column of numbers. $\endgroup$
    – Jack LaVigne
    Oct 27, 2015 at 23:11
1
$\begingroup$

I think I understand now, how to solve the question partially based on the @ MarcoB advice. Observing that the billhead ends up with "cc/g\n" let us first delete the billhead assuming that text is the freshly imported string:

text1 = StringDelete[text, __ ~~ "cc/g\n"]

This yields the following

     16.4540           4.0763e-02       4.9548e+01      1.9462e-02       \
2.3656e+01       7.3634e-01       8.9503e+02
      18.7036           1.1419e-01       1.2807e+02      3.0536e-02   \
    3.2653e+01       1.3133e+00       1.4043e+03
      21.3054           2.0390e-01       2.1228e+02      3.2050e-02   \
    3.0086e+01       1.5700e+00       1.4738e+03
      24.3856           2.1418e-01       2.2071e+02      3.0579e-03   \
    2.5080e+00       1.7143e-01       1.4060e+02
      26.7185           2.1639e-01       2.2236e+02      1.6974e-03   \
    1.2706e+00       1.0440e-01       7.8151e+01
      28.1997           2.1876e-01       2.2404e+02      1.4286e-03   \
    1.0132e+00       9.2736e-02       6.5771e+01
      29.9055           2.2086e-01       2.2545e+02      1.1975e-03   \
    8.0084e-01       8.2435e-02       5.5130e+01
      31.7795           2.2299e-01       2.2679e+02      1.0700e-03   \
    6.7342e-01       7.8275e-02       4.9261e+01
      33.8090           2.2493e-01       2.2794e+02      9.3685e-04   \
    5.5420e-01       7.2909e-02       4.3130e+01
      36.3742           2.2723e-01       2.2920e+02      7.5219e-04   \
    4.1359e-01       6.2962e-02       3.4619e+01
      38.8557           2.2862e-01       2.2991e+02      7.2847e-04   \
    3.7496e-01       6.5162e-02       3.3541e+01
      41.8657           2.3096e-01       2.3103e+02      5.6815e-04   \
    2.7142e-01       5.4726e-02       2.6143e+01
      45.1997           2.3233e-01       2.3164e+02      5.3931e-04   \
    2.3864e-01       5.6115e-02       2.4830e+01
      49.3039           2.3460e-01       2.3256e+02      4.0052e-04   \
    1.6247e-01       4.5419e-02       1.8424e+01
      53.9869           2.3595e-01       2.3306e+02      3.6589e-04   \
    1.3555e-01       4.5465e-02       1.6843e+01
      60.1244           2.3830e-01       2.3384e+02      2.7388e-04   \
    9.1106e-02       3.7853e-02       1.2591e+01
      67.5290           2.3959e-01       2.3423e+02      2.0645e-04   \
    6.1143e-02       3.2078e-02       9.5004e+00
      75.6420           2.4133e-01       2.3469e+02      1.7462e-04   \
    4.6170e-02       3.0370e-02       8.0299e+00
      87.3612           2.4310e-01       2.3509e+02      1.3141e-04   \
    3.0085e-02       2.6382e-02       6.0398e+00
     102.3268           2.4499e-01       2.3546e+02      1.1469e-04   \
    2.2417e-02       2.6965e-02       5.2704e+00
     130.3970           2.4721e-01       2.3580e+02      5.5938e-05   \
    8.5796e-03       1.6665e-02       2.5560e+00
     166.7821           2.4845e-01       2.3595e+02      3.7325e-05   \
    4.4759e-03       1.4287e-02       1.7132e+00
     248.5847           2.5013e-01       2.3608e+02      1.2880e-05   \
    1.0363e-03       7.2000e-03       5.7928e-01
     419.7893           2.5065e-01       2.3611e+02      2.4805e-06   \
    1.1818e-04       2.3458e-03       1.1176e-01

Now the idea of MarcoB can be applied:

SemanticImportString[text1] // Normal

returning this:

    {{16.454, 0.040763, 49.548, 0.019462, 23.656, 0.73634, 
  895.03}, {18.7036, 0.11419, 128.07, 0.030536, 32.653, 1.3133, 
  1404.3}, {21.3054, 0.2039, 212.28, 0.03205, 30.086, 1.57, 
  1473.8}, {24.3856, 0.21418, 220.71, 0.0030579, 2.508, 0.17143, 
  140.6}, {26.7185, 0.21639, 222.36, 0.0016974, 1.2706, 0.1044, 
  78.151}, {28.1997, 0.21876, 224.04, 0.0014286, 1.0132, 0.092736, 
  65.771}, {29.9055, 0.22086, 225.45, 0.0011975, 0.80084, 0.082435, 
  55.13}, {31.7795, 0.22299, 226.79, 0.00107, 0.67342, 0.078275, 
  49.261}, {33.809, 0.22493, 227.94, 0.00093685, 0.5542, 0.072909, 
  43.13}, {36.3742, 0.22723, 229.2, 0.00075219, 0.41359, 0.062962, 
  34.619}, {38.8557, 0.22862, 229.91, 0.00072847, 0.37496, 0.065162, 
  33.541}, {41.8657, 0.23096, 231.03, 0.00056815, 0.27142, 0.054726, 
  26.143}, {45.1997, 0.23233, 231.64, 0.00053931, 0.23864, 0.056115, 
  24.83}, {49.3039, 0.2346, 232.56, 0.00040052, 0.16247, 0.045419, 
  18.424}, {53.9869, 0.23595, 233.06, 0.00036589, 0.13555, 0.045465, 
  16.843}, {60.1244, 0.2383, 233.84, 0.00027388, 0.091106, 0.037853, 
  12.591}, {67.529, 0.23959, 234.23, 0.00020645, 0.061143, 0.032078, 
  9.5004}, {75.642, 0.24133, 234.69, 0.00017462, 0.04617, 0.03037, 
  8.0299}, {87.3612, 0.2431, 235.09, 0.00013141, 0.030085, 0.026382, 
  6.0398}, {102.327, 0.24499, 235.46, 0.00011469, 0.022417, 0.026965, 
  5.2704}, {130.397, 0.24721, 235.8, 0.000055938, 0.0085796, 0.016665,
   2.556}, {166.782, 0.24845, 235.95, 0.000037325, 0.0044759, 
  0.014287, 1.7132}, {248.585, 0.25013, 236.08, 0.00001288, 0.0010363,
   0.0072, 0.57928}, {419.789, 0.25065, 236.11, 2.4805*10^-6, 
  0.00011818, 0.0023458, 0.11176}}

I thank you for the discussion.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.