How to write output to an external text file in addition to the standard output stream?

Question

How to write output to an external text file in addition to the standard output stream, i.e. a notebook for a session with FE and stdout for command-line session?

Answer 1

I expected this to work but it apparently only writes Print output:

AppendTo[
  $Output, 
  OpenWrite["out.txt", FormatType -> OutputForm]
]

A workaround using $PrePrint:

output = OpenWrite["out.txt", FormatType -> OutputForm];

$PrePrint = (Write[output, #]; #) &;