# Understanding the discrepancy between txt file import and export times, and possibly speeding up Import

I attempted to import a very short .txt file string using the Import command:

Import["C:\\TestFile.txt"] // AbsoluteTiming

Out: {0.020001, "Test"}


Why does this take 20 milliseconds? Export is about an order of magnitude faster for the same string:

Import["C:\\TestFile.txt","Test"] // AbsoluteTiming

Out: {0.002000, "C:\\test.txt"}


Are there any hidden options for Import like the following (concerning TIF files) from a previous question of mine Slow import of multigigabyte TIF image stacks?:

ImageImportExportDumpImageReadTIFF


Quoting the user cormullion from my previous question: "For the OP's single frame, it's 0.015 for Import, but 0.000745 for ImageReadTIFF on Mac/M9. That might be worth having."

• Import comes with a lot of overhead. Try for example Import["testfile.txt", "Text"] // AbsoluteTiming versus just Import["testfile.txt"] and that removes just one test Import has to make. Then try ReadList["testfile.txt", "String"] // AbsoluteTiming which hasn't got the same sort of overhead. Also try Trace[Import["testfile.txt"]] to see exactly the internal functions used and the kind of tests performed. – C. E. Sep 7 '13 at 13:14

In the interest of reducing the number of questions that have been more or less thoroughly "answered in comments", I take the liberty to transform Pickett's insightful comment into an answer:

Import comes with a lot of overhead. Compare for example the following two snippets:

RepeatedTiming[
ClearSystemCache[];
Import["testfile.out"];
]

RepeatedTiming[
ClearSystemCache[];
Import["testfile.out", "Text"];
]

(*Out:
{0.329, Null}
{0.14, Null}
*)


Simply providing Import with information on the format of the data removes a lot of content testing overhead!

You can compare the performance of Import with that of functions that don't need to carry out as much testing on their input, e.g. ReadList:

RepeatedTiming[
ClearSystemCache[];

Also an interesting idea is to look at Trace[Import["testfile.out"]] to see exactly the internal functions used and the kind of tests performed.