is there something like a "foreach" mode from the command line? Like in perl?

Question

With perl we can run perl -p . That wraps a foreach loop around the code and executes the code on the file. What I'm looking for is something like

StringReplace[filename,"\t"->","]

and then I get a new file in which the string replace happened (the above would convert a .tsv to a .csv). Doesn't have to be StringReplace, can be anything that takes a line of stuff as input. Total[filename] would total all numbers in the rows of the file.

I do NOT want the whole file in kernel memory!

also like sed on Linux (stream editor -- works line by line)

That is like "scripting" the M command line-by-line (during MS-DOS we used to call that "batch mode" -- really means line-by-line).

Answer 1

I think you're looking for the -linewise option to wolframscript.

Here's my little script, tsv2csv

#!/usr/bin/env wolframscript -linewise

Print@StringReplace[$ScriptInputString,"\t"->","]

I tried it, via ./tsv2csv < in > out on the file in

a   b   c
d   e   f
g   h   i

and got

a,b,c
d,e,f
g,h,i

I believe the script is essentially called 3 times, because if I replace the StringReplace[...] with "Foo" the file out contains 3 lines, each says Foo.

Answer 2

This is not robust but should be enough to get you started.

ClearAll[applyToFile]
applyToFile[filename_, typeSpec_, func_, batchSize_Integer?Positive] :=
 Module[{read = OpenRead[filename], write = OpenWrite[], readBach, 
   writeFileName},
  readBach = ReadList[read, typeSpec, batchSize];
  While[readBach =!= {},
   readBach = func /@ readBach;
   WriteLine[write, #] & /@ readBach;
   readBach = ReadList[read, typeSpec, batchSize]
   ];
  Close@read;
  Close@write
  ]

Then

SeedRandom[123]
fn = Export[FileNameJoin[{$TemporaryDirectory, "nums.txt"}], RandomInteger[20, {5, 5}], "CSV"];
FilePrint[fn]

14,18,0,4,7
6,7,10,11,6
10,12,9,17,0
9,11,16,15,6
12,5,15,12,20

and

newFileName = applyToFile[fn, Record, StringReplace["," -> "\t"], 3];
FilePrint[newFileName]

14    18  0   4   7
6   7   10  11  6
10  12  9   17  0
9   11  16  15  6
12  5   15  12  20

Hope this helps.