Speeding up export of a matrix to a dat file

Question

I'm trying to export a 1000*100 matrix of numbers to a .dat file using Export (I want the numbers to be separated by a tab) and it's very slow, it takes several minutes, whereas when I export to an Excel file it takes a few seconds.

Is there a way to speed this up ?

Also how can I control the number of decimals I want to export ?

Edit
I found the issue, this happens when I export my dat file on a network drive (at work), the export to an Excel file and a dat file must be implemented in different ways that produce different behaviours. Thank you though for the different interesting answers.

Answer 1

I suggest you watch the presentation BigData: Demystifying Large Datasets in Mathematica by Nick Lariviere. He discusses the various possibilities quite clearly.

Answer 2

To answer your second question, to control the number of decimals of the exported data you could change the accuracy of your data using SetAccuracy. For example if you only want to export the first two decimals of each entry you could do something like

data = RandomReal[20, {1000, 100}];
Export["test.dat", SetAccuracy[data, 3]]

Answer 3

I cannot reproduce your behavior here. For a $1000\times 1000$ it requires only some seconds here

data = Table[N[i/j], {j, 1000}, {i, 1000}];
AbsoluteTiming[
 Export["tmp/matrix.dat", data, "Table", "FieldSeparator" -> {"\t"}]]

(*
  {13.442387, "tmp/matrix.dat"}
*)

If your separator for numbers is always a tab, then clearly the way @Szabolcs pointed out in his comment should be the preferred one.

Export["tmp/matrix.dat", data, "TSV"]

My sample shows, that you can basically use every desired line/field separator you like.

Heike gave the answer to your second question. I would like to point out, that you might lose information because real numbers are usually stored with $MachinePrecision. When you use now SetAccuracy or SetPrecision you cut them off. There is usually only one situation where you want this: When you look at the numbers. I suspect you are looking at a $1000\times 100$ matrix, so why do you want to cut your numbers?