2
$\begingroup$

Summary: The results of a grep command on fixed text files SHOULD be extremely repeatable, but by the time they arrive in Mathematica they are not. When rapidly reading many shell commands, Mathematica's answers are wildly inconsistent.

The first line below discovers 9 small fixed text files. The second one repeatedly requests to find the word "fini" in the files and to print the line containing it. The Length function merely counts the number of words found.

gameNames=FileNames["game000*"];
Table[Length[ReadList["!grep fini "<>#,Word]]& /@ gameNames,6] //TableForm

This should produce a table of 6 lines where each column is all the same single consistent value. But instead it produces output like this:

4   4   4   4   4   4   0   0   0
0   4   0   0   0   0   0   0   0
0   0   4   4   4   0   4   0   0
0   0   0   4   0   0   0   0   0
0   4   4   0   0   0   4   4   0
4   0   0   0   0   4   4   0   0

Most columns here are erratic, which means the grep command is erratic, or the ReadList is. Which one? Why? What can be done about it?

In many executions of this code, the top left "4" is always the same (perhaps because it is the first in a rapid burst of executions), but most other entries seem capricious. The first 8 files each have a line in them containing 4 words, one of which is the word "fini" -- so the first 8 columns should be all 4's; the last file has no such line, so it should always be a column of 0's, which it has been in all runs observed.

Here is the data as obtained on a Unix Terminal (the absence of the 9th file in this list is what signals its lack of the word "fini" in it):

MacBook-Pro:BNS001-KPpairs me$ grep fini game000*
game0001.data:fini time limit expired
game0002.data:fini time limit expired
game0003.data:fini time limit expired
game0004.data:fini time limit expired
game0005.data:fini time limit expired
game0006.data:fini time limit expired
game0007.data:fini time limit expired
game0008.data:fini time limit expired

I posted a similar question several days ago, and therein I suggested that it might be an iCloud issue. The example given above, however, is running on files that live on a MacBook running Catalina below my /Users account not inside the Documents folder. No file or directory in the path has a little cloud symbol beside it in the Finder display. So I believe that possibility does not hold here.

$\endgroup$
9
  • $\begingroup$ Can you put the output of grep fini game000* into your post as well? Might help figure out what's going on. $\endgroup$
    – Carl Lange
    Commented Sep 2, 2020 at 13:11
  • $\begingroup$ done, @Carl Lange. Thanks for your interest. $\endgroup$
    – Carib John
    Commented Sep 2, 2020 at 13:48
  • 2
    $\begingroup$ Very odd. Looks like a bug. Equal @@@ Table[Length[ReadList["!ls", "Word"]], {5}, {5}] should always be a list of True. Sometimes it is not. $\endgroup$ Commented Sep 2, 2020 at 18:47
  • 1
    $\begingroup$ @RohitNamjoshi I could reproduce the issue with your command under macOS not under Windows (replacing ls with dir) $\endgroup$ Commented Sep 2, 2020 at 20:18
  • $\begingroup$ Yes, @Rohit, your formulation failed on my machine too. The flaw in all examples so far is that the ReadList either gets it all right, or sees nothing; I've seen no intermediate cases. $\endgroup$
    – Carib John
    Commented Sep 2, 2020 at 20:55

1 Answer 1

1
$\begingroup$

I have no explanation for the failure of ReadList, but an apparent workaround is to use Import["!grep fini "<>#,"Text"] instead of ReadList["!grep fini "<>#,Word]. I am not going to guarantee anything, but it seems stable.

$\endgroup$
1
  • $\begingroup$ From my experiments, adding a small pause between each ReadList call also helps. Pause[0.1] or something. Maybe useful, although maybe Import is completely sufficient. $\endgroup$
    – ktm
    Commented Sep 25, 2020 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.