How to Import[...] a datafile from a file created by OpenAppend[...]

Question

I created a datafile as follows:

ESTgrowth = FileNameJoin[{"Filepath", "ESTgrowth"}];
open = OpenAppend[ESTgrowth];
Write[open, newdata];  (*"newdata" is a list of data appended to "ESTgrowth" *)
Close[open];

This code appends data stream as follows:

{-17.071, 137.567, 16.554, 0.65}
{-16.873, 136.138, 16.934, 0.7}
{-15.397, 135.738, 16.894, 0.7}
{-14.7, 135.982, 15.9, 0.6}

Then I tried to import the data file to present it using TableForm[...]. Of course,

Import[ESTgrowth, "Table"]

would NOT work because the data stream consists of lists of data only. My question is how to automatically put the data stream into the following format:

{ 
  {-17.071, 137.567, 16.554, 0.65},
  {-16.873, 136.138, 16.934, 0.7},
  {-15.397, 135.738, 16.894, 0.7},
  {-14.7, 135.982, 15.9, 0.6}
}

for TableForm[...] to work as expected.

Screenshot of my dataset with Extra commas placed in fixed positions in each row.

Screenshot of NOT working code.

Answer 1

You are better off removing the MMA formatting completely, and then relying on ImportString's euristics:

text = Import["yourFile.dat", "Text"];
ImportString@StringReplace[text, {"{" -> "", "}" -> ""}]

(*Out:
{
 {-17.071`, 137.567`, 16.554`, 0.65`},
 {-16.873`, 136.138`, 16.934`, 0.7`},
 {-15.397`, 135.738`, 16.894`, 0.7`},
 {-14.7`, 135.982`, 15.9`, 0.6`}
} 
*)

Really, though, it would be best to plan ahead and write a more sensible format to file (say, a CSV?) in the first place.

Answer 2

This works on V 12.1 on windows

SetDirectory[NotebookDirectory[]]
open = OpenAppend["ESTgrowth.txt", FormatType -> InputForm];
Do[
  newdata = RandomInteger[{0, 10}, 4];
  Write[open, newdata],
  {n, 1, 3}
  ];
Close[open];

Now the file looks like this

To read it back, we must make a list of lists. This is done by replacing each "\n" with "," and adding "{" at start and adding "}" at end of the string. Only after that, we can convert it to Mathematica experssion.

dataBack = Import["ESTgrowth.txt"];
dataBack = StringReplace[dataBack, "\n" -> ","];
dataBack = StringInsert[dataBack, "{", 1];
dataBack = StringInsert[dataBack, "}", -1];
dataBack = ToExpression[dataBack]

(*    {{3, 9, 1, 2}, {8, 10, 10, 10}, {0, 2, 0, 5}}  *)

If we now run the above code all over again, it still works, and the file now looks like this

And data is read back OK as

  {{3, 9, 1, 2}, {8, 10, 10, 10}, {0, 2, 0, 5}, {0, 7, 5, 6}, {9, 8, 8, 
     8}, {8, 7, 8, 2}}

Update

For V 11.3, it has an extra "\n" at start of file. So for V 11.3 the following should work

open = OpenAppend["ESTgrowth.txt", FormatType -> InputForm];
Do[
  newdata = RandomInteger[{0, 10}, 4];
  Write[open, newdata],
  {n, 1, 3}
  ];
Close[open];

Now

dataBack = Import["ESTgrowth.txt"];
dataBack = StringReplace[dataBack, "\n" -> ","];

(*this line below only needed for V 11.3 *)
dataBack = StringDrop[dataBack, 1]; (*delete the extra "," we added*)

dataBack = StringInsert[dataBack, "{", 1];
dataBack = StringInsert[dataBack, "}", -1];
dataBack = ToExpression[dataBack]

Gives

 {{1, 8, 8, 7}, {0, 7, 8, 0}, {9, 10, 6, 6}}

Update

It looks like OP is till seeing an issue on V 11.3, but I am not able to duplicate it. Here are the steps again showing result of each step. On windows 10.

Here is showing step-by-step the above on V 11.3

open = OpenAppend["ESTgrowth.txt", FormatType -> InputForm];
Do[
  newdata = RandomInteger[{0, 10}, 4];
  Write[open, newdata],
  {n, 1, 3}
  ];
Close[open];

dataBack = Import["ESTgrowth.txt"];
InputForm[dataBack]

dataBack = StringReplace[dataBack, "\n" -> ","];
InputForm[dataBack]

dataBack = StringDrop[dataBack, 1]; (*delete the extra "," we added*)
InputForm[dataBack]

dataBack = StringInsert[dataBack, "{", 1];
InputForm[dataBack]

dataBack = StringInsert[dataBack, "}", -1];
InputForm[dataBack]

dataBack = ToExpression[dataBack];
InputForm[dataBack]

gives

 {{3, 3, 1, 2}, {9, 6, 7, 8}, {6, 1, 10, 6}}