I have a single flat directory with over a million files. I just wanted to take a sample of the first few files but FileNames doesn't include a "only the first n" option, and so it took over a minute:

enter image description here

Is there a faster way?


3 Answers 3


Using OS shells commands seems to be much faster, although their output will need some massaging to obtain only the file names.

For instance, the following works quite well on my system (Win7-64):

Import["!dir /b /a-d C:\\Windows\\*", "Text"] // StringSplit[#, "\n"] &

This command takes 0.11s to execute; for comparison the corresponding FileNames expression (FileNames["*", "C:\\Windows\\"]) takes 0.33s. The difference is even more pronounced with larger/deeper directories.

In the dir command above, /a-d selects files that are not directories; /b produces minimal output which is easier to parse. If you want to traverse subdirectories, you can use the /s option.

  • $\begingroup$ I have OSX 10.11 $\endgroup$
    – M.R.
    Commented Jun 14, 2016 at 18:37
  • $\begingroup$ @M.R. OK. So have you tried replacing the corresponding Unix-style shell commands in the Import expression? Does that not work? $\endgroup$
    – MarcoB
    Commented Jun 14, 2016 at 18:44
  • $\begingroup$ @M. R., then you might want to look up the man page for ls. $\endgroup$ Commented Jun 14, 2016 at 23:54
  • $\begingroup$ i think youll need find to list files-not-directories. ls will do fine for a specific extension though $\endgroup$
    – george2079
    Commented Jun 15, 2016 at 7:05
  • $\begingroup$ @george, or use the -p switch in ls. $\endgroup$ Commented Jun 15, 2016 at 10:46

New function in Mathematica 11 FileSystemMap with option MaxItems (documentation) can be useful here.

dir = "C:\\Users\\Alexey\\Documents";
n = 10;
f = FileSystemMap[#&, dir, MaxItems -> n] // Keys;
  • $\begingroup$ Try this on a large directory with, it hangs when you specify a recursive depth. n = 10; f = FileSystemMap[#&, dir, Infinity, MaxItems -> n] // Keys; $\endgroup$
    – M.R.
    Commented Sep 7, 2016 at 17:55
  • $\begingroup$ @M.R. You are correct. This works not very good on a large directory. FileSystemMap and FileSystemScan are much slower than FileNames. It's very strange. $\endgroup$ Commented Sep 10, 2016 at 3:15

a linux version using find to list files and head to take the first n.

dir = "/path"
          "find "<>dir<>" -maxdepth 1 -type f | head -10"

note find does not return files in any canonical order, you can sort for example, add | sort in front of | head

Note FileNames[][;;10]] is a good bit faster by my testing, but I don't have a folder with a million files to test it on.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.