How do I reliably read from stdin using wolframscript?

Question

I am trying to direct a stream of data to a wls script in a command line environment. Wolframscript provides the $ScriptInputString symbol to read stdin. However, when the input is large, no input is apparently read.

How do I read from the standard input for large sizes?

---

Minimal (not) working example:

for i in $(seq 0 $SIZE); do echo $i; done | wolframscript -print -c \$ScriptInputString

When $SIZE=100000 (one hundred thousand), outputs all integers from 0 to 1000000. When $SIZE=1000000 (one million), outputs nothing.

$Version is 11.3.0 for Linux x86 (64-bit) (March 7, 2018)

Answer 1

The implementation of $ScriptInputString in wolframscript is buggy. I have found that providing !cat to an import function (ReadString, Import, ...) is a more stable way of reading from stdin.

When using wolframscript, this approach only works in cases where $ScriptImportString doesn't, presumably because the stream is already consumed otherwise.

A combination of the two works for both small and large inputs in my tests:

stdin = $ScriptInputString <> Import["!cat","String"]