# How to export data files using specific format

Very often, I have to export data files from Mathematica. These output files contain several columns and hundreds of lines with data. For the sake of the question let's create a sample list

data = Table[{RandomReal[i], RandomReal[i^10]}, {i, 0, 100, 0.1}];


Then, I export the data in an external file using the following command

Export["data.out", data, "Table"];


Now let's take a look at the output file. I present here only the first ten lines of data, but of course the whole file is easily reproducible.

0.  0.
0.01599700768300292    9.144414172338937e-12
0.05264806593747054    5.191535295682516e-8
0.26762136311267287   2.6644991730537336e-6
0.01121545335019451     0.000014373686962031977
0.05282872336357303   0.0005555775821525191
0.39492299878320175   0.0005639987736085206
0.6483050598753638    0.016687867077835535
0.5871008451356092    0.029518547448633307
0.2613882661983659    0.046494918800056996
..................    ....................


My problem is that the format of the data is very bad. When I create external output files in FORTRAN I can control the exact format. So, I am wondering: Is this possible in Mathematica? Can I control the number of decimal digits of the data, using exponential form or even the gap between the two columns? In FORTRAN, I would use this format 2(4X,E16.8), meaning that the data would be in exponential form (E), having total 16 digits in total from which the 8 are decimal and finally the gap between the two columns is 4 blank characters.

EDIT:

Following cormullion's instructions the exported data files look like this

0.0000000000000000E     0.0000000000000000E
0.0292206329521133E     3.5660981240075680E-12
0.0110286543632250E     7.1403083420409010E-8
0.1512525645232069E     3.4764180261918010E-8
0.1802104544376126E     0.0000150833977857E
0.1909866023262902E     0.0004442874170494E
0.1977662892883431E     0.0047809317792338E
0.1043698170574586E     0.0116230093104291E
0.2452055184319113E     0.0756524674940767E
0.4203148306922685E     0.0682570346613104E
0.4640953468992068E     0.6204923184434734E
0.4361142873366830E     0.8423302837693910E
0.3853270077185619E     1.9278084597534120E
0.9173409732335570E     1.9391384326481460E
0.2309798126928349E     6.9819944783354480E
0.9064707972602210E     55.0683061788128700E
1.1871007725662080E     47.5205793705051100E
0.0657321188104301E     165.0478398165293000E
0.6501851799068231E     261.9422532326231000E
0.1976942785179729E     83.6983691851117000E
0.1781317179022484E     877.1531182356240000E
0.9603071406080800E     610.9760718811238000E
1.2678697679679910E     2107.5934446883950000E
1.2555519112242100E     695.2293658225135000E
1.4503489860688380E     6300.5931889760220000E
2.3698084216652830E     9336.5466490215000000E
0.9495735185410430E     8100.1159435400300000E
2.3382755917778770E     18471.7912516294500000E
1.3595555295319900E     27021.6788394801700000E
1.1450366450215240E     20223.3331979412600000E
0.6496852979125343E     14269.5720554042100000E
1.4074545584871190E     22413.5903375179000000E
1.8790499719570280E     85437.5672819188000000E
3.0597774075552670E     37024.4278765415400000E
3.2694586701530210E     99567.9453873553000000E
2.7064643220362330E     226071.8308196532000000E
...................     ........................


As everyone can observe, there are some inconsistencies regarding the format

(i). The exponential part (E) should always have the same pattern; two digits (i.e. E-08 not just E-8) and when is zero should be E+00

(ii). In some cases, the second column contains some additional (more than 16) decimal digits (in fact they are unwanted zeros).

I filtered this data file using a very simple FORTRAN code and here is the corresponding output

0.0000000000000000E+00    0.0000000000000000E+00
0.2922063295211330E-01    0.3566098124007568E-11
0.1102865436322500E-01    0.7140308342040901E-07
0.1512525645232069E+00    0.3476418026191801E-07
0.1802104544376126E+00    0.1508339778570000E-04
0.1909866023262902E+00    0.4442874170494000E-03
0.1977662892883431E+00    0.4780931779233800E-02
0.1043698170574586E+00    0.1162300931042910E-01
0.2452055184319113E+00    0.7565246749407670E-01
0.4203148306922685E+00    0.6825703466131040E-01
0.4640953468992068E+00    0.6204923184434734E+00
0.4361142873366830E+00    0.8423302837693910E+00
0.3853270077185619E+00    0.1927808459753412E+01
0.9173409732335570E+00    0.1939138432648146E+01
0.2309798126928349E+00    0.6981994478335448E+01
0.9064707972602210E+00    0.5506830617881287E+02
0.1187100772566208E+01    0.4752057937050511E+02
0.6573211881043010E-01    0.1650478398165293E+03
0.6501851799068231E+00    0.2619422532326231E+03
0.1976942785179729E+00    0.8369836918511170E+02
0.1781317179022484E+00    0.8771531182356240E+03
0.9603071406080800E+00    0.6109760718811237E+03
0.1267869767967991E+01    0.2107593444688395E+04
0.1255551911224210E+01    0.6952293658225135E+03
0.1450348986068838E+01    0.6300593188976022E+04
0.2369808421665283E+01    0.9336546649021500E+04
......................    ......................


Now, everything is OK! Well, as Xerxes pointed out, why care what the file looks like as long as the next computer program knows how to parse it? The answer is very simple. Because scientists love neatness and order in their work and they hate sloppy jobs! Moreover, I refuse to believe, that a sophisticated and modern software like Mathematica cannot export data using predefined format.

-
Investigate FortranForm, NumberForm, ScientificForm, EngineeringForm functions. –  murray Feb 10 '13 at 22:09
Call me Fortran-illiterate, but I cannot tell what "having total 16 digits in total from which the 8 are decimal" means. I wonder if this fiddly problem really needs solving or if you've just gotten used to how Fortran outputs things. Is a human really going to read hundreds of 16-digit numbers? If not, why care what the file looks like as long as the next computer program knows how to parse it? –  Xerxes Feb 10 '13 at 22:13
@Xerxes Well, that still does not answer the question (which I'm almost sure is a duplicate, I just can't find it now). Using the some arguments, one could just dump everything as binary... –  Ajasja Feb 10 '13 at 22:31
@Ajasja: Just a comment, not an answer. :-) Binary is a great solution if you know your next parser can read it. ASCII text numbers divided by a few distinct kinds of whitespace or punctuation can be read by just about anything, which is more robust if you're not 100% sure what might read it next. –  Xerxes Feb 10 '13 at 22:44
can you give a example of a number that have "16 digits in total from which the 8 are decimal" ? –  andre Feb 10 '13 at 23:12

This is a variant on Andre's solution (worked out before I noticed!). Anyway this takes the FORTRAN descriptor values as arguments.

f77Eform[x_?NumericQ, fw_Integer, ndig_Integer] := Module[{sig, s, p, ps},
{s, p} = MantissaExponent[x];
{sig, ps} = {ToString[Round[10^ndig Abs[s]]], ToString[Abs[p]]};
StringJoin @@ Join[
Table[" ", {fw - ndig - 7}],
If[x < 0, "-", " "], {"0."}, {sig},
Table["0", {ndig - StringLength[sig]}], {"E"},
If[p < 0, {"-"}, {"+"}],
Table["0", {2 - StringLength[ps]}], {ps}]]

f77Eform[Pi, 16, 8] ->       "  0.31415927E+01"
f77Eform[-Sqrt[2], 16, 8] -> " -0.14142136E+01"
f77Eform[10^-4, 16, 8]    -> "  0.10000000E-03"
f77Eform[-10^6, 16, 8]    -> " -0.10000000E+07"


sanity check:

Max[Table[(Abs[{Read[StringToStream[f77Eform[#, 22, 14]], Number]/# -
1}]) &[RandomReal[{-1, 1}] 10^RandomReal[{-5, 5}]] , {1000}]]   -> 4.64073*10^-14


I must say as someone who works with Fortran quite a lot I have never needed to do this. Fortran can perfectly well read more generally formatted data. Unless you are stuck with legacy Fortran code that uses fixed field formatting for input don't bother.

-
hold on..bug fixing.. –  george2079 Feb 13 '13 at 16:28
fixed -- forgot the "+/-" on the E.. –  george2079 Feb 13 '13 at 16:39

Mathematica has so many output and formatting options, it's quite hard to piece all the bits together sometimes. Here's something I've managed to build by looking at the extensive documentation:

formattedData = Partition[
StandardForm@
#,
{16, 16},
NumberFormat -> (StringJoin[#1, "E", #3] &)]
& /@ Flatten@data, 2];


This applies a format to each number and turns the whole list back into a table.

{{ 0.0000000000000000E, 0.0000000000000000E},
{ 0.0688287941758228E, 7.8480265933168900E-11},
{ 0.0075608357710283E, 9.3849677784910300E-8},
{ 0.2817481779372621E, 4.9262973903383610E-6},
...


Now you can export it, using the "FieldSeparators" option for the "Table" format:

Export["out.dat", formattedData, "FieldSeparators" -> "    "]


and your file is now in two columns, separated by four spaces:

 0.0000000000000000E     0.0000000000000000E
0.0688287941758228E     7.8480265933168900E-11
0.0075608357710283E     9.3849677784910300E-8
0.2817481779372621E     4.9262973903383610E-6
...


However, to get precisely 16 digits in total, with 8 after the decimal point, you'll need to ask another question!

-
I followed your instruction but still there are some unresolved issues. See my EDIT for more details. Anyway, I approved your answer! Could you post a link for the documentation about the format? Specifically, the one mentioning your formattedData code. I looked but I did not find anything so far. –  Vaggelis_Z Feb 12 '13 at 15:30
@Vaggelis_Z Thanks for accept, although to be honest I was hoping someone would post something less clumsy. I think there is probably a better way, but I've not used these functions before. –  cormullion Feb 12 '13 at 15:51

Unfortunately I have no Mathematica access right now, but I would like to point you to some links which I think might be helpful in solving your issue:

Modifying the appearance of numbers: http://reference.wolfram.com/mathematica/ref/NumberForm.html

-

I find NumberForm, FortranForm ... not adequate for this problem.

This is how I would do it :

Treatment for one number :

toEString[dat_] :=
If[dat == 0.,
"0.0000000000000000+E00",
MantissaExponent[dat] //
With[{mantissa = #[[1]], exponent = #[[2]]},
{
If[mantissa < 0, "-", ""],
"0.",
ToString /@
If[exponent < 0, "-E", "+E"],
ToString /@ IntegerDigits[exponent, 10, 2]
} //
Flatten //
StringJoin] &
]


Concatenation of the whole in a string :

StringJoin[
StringJoin[
toEString[#[[1]]] <> "   " <> toEString[#[[2]]] <> "\n"] &  /@
data
]

(*
0.0000000000000000+E00   0.0000000000000000+E00
0.6234044134173835-E01   0.7360326360478650-E12
0.1087127423322964+E00   0.6627601340696958-E07
0.2629754862938250+E00   0.4328843827282185-E05
0.3389585889389961+E00   0.4500485245522299-E05
0.2113663038718073+E00   0.6037547118023614-E03
*)


and then export the string as text (with Export[fileName,the String,"Text"]).

I don't know what you expect if some numbers are < 0.

-
@Vaggelis_Z another solution with more control of what's happening –  andre Feb 12 '13 at 21:00
note the sign should follow the E .. (simple fix "-E" -> "E-" ) –  george2079 Feb 13 '13 at 16:42