Mathematica Stack Exchange is a question and answer site for users of Mathematica. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've written a function to pull in the useful column of a large data file:

ImportNthColumn[fname_, n_] := With[{
  command = "!awk '{ print $" <> ToString[n] <> " }' "
  Flatten@Import[command <> fname, "Table"]

and I'd like to apply this function to pull in data from every subdirectory, like so:

ImportNthColumnFromDirectories[dirs_, fname_, n_] := With[{
  targets = Map[# <> "/" <> fname&, dirs]
  Map[ImportNthColumn[#,n]&, targets]

When I use ImportNthColumnFromDirectories with a serial map, everything works perfectly. When I use ParallelMap, however, the data in the first and last elements of dirs import correctly but all other iterations return a zero-length list. What could cause this strange behavior?

share|improve this question
Can you give a reproducible example? Without your data files it's hard to say much of anything. One thing you could do is use head on each of the files to make smaller versions that you could then upload. – Taliesin Beynon Jul 30 '14 at 21:09

Documentation say:

Import["!prog","format"] imports data from a pipe.

I suggest that ParallelMap sent many requests to external program in one time and it return errors, but Mathematica interpret errors as empty list.

PS: sorry for my english

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.