I have about 1 million .txt text files in a directory which I would like to all concatenate in one single text file (separated by a newline character "\n"). Because of the number of files and their size I would rather not like to read everything in memory first. So I was wondering - what is the fastest way to do this in Mathematica, without having to resort to read everything in memory first?
cheers, Tom
cat *.txt > hugefile.txt
is sufficient to join the files. $\endgroup$OpenAppend
. Basically I'm thinking you could append one file at a time. Or you might use Unix tools from within Mathematica. $\endgroup$Run["cat *.txt > hugefile.txt"]
. $\endgroup$