Bug fixed in 9.0.0
Update There is now an official response from Wolfram Support on this issue.
Having recently installed Mathematica version 9 64-bit on Windows 7, I keep running into Self-Test Errors evaluating certain expressions. For example, if I evaluate:
Piecewise[{{0, x < 10}, {1, True}}]
then this message pops up:
INTERNAL SELF-TEST ERROR: NoteFont2|c|1982
The output cell did not render properly and, when I hover over the cell, the following additional message pops up:
The stretchable character 0xf361 in the Mathematica5Mono font (size 13) has a repeating piece (0xb2) that is zero size.
The same SELF-TEST error pops up if I type ESCpwESC in an input cell.
This error occurrs on two different Windows 7 machines, installing the Standard Edition on one and the Home Edition on the other. Both machines also have versions 7 and 8 installed. On the Standard Edition machine, it is very likely that Mathematica had been run many times before installation after the last reboot. But on the Home Edition machine the installation was the first thing done after power-on.
Interestingly, my pre-existing installations of version 7 and version 8 both exhibit this behaviour now. They worked prior to installing version 9.
How can I fix this?
Update
Some users report getting a Windows "Blue Screen of Death" (BSOD) performing these or similar operations.