I have a VerificationTest that is expected to generate some messages. It runs inside of code that uses Check to Abort[] on any messages generated in non-test code.

This construction does not work as intended, however: The Messages generated inside of VerificationTest seem to leak and trigger Check (even though $MessageList is empty?).

This code:

Check[Echo@VerificationTest[Message[a::b]; True, True, {a::b}]; Echo@$MessageList, $Failed]

makes the test succeed, but Check returns $Failed, while this code:

Check[Quiet@Echo@VerificationTest[Message[a::b]; True, True, {a::b}]; Echo@$MessageList, $Failed]

makes the test fail, because the effect of Quiet causes the message to go unregistered within VerificationTest (I think).

What can I do here?

Is there maybe even a bug with one of Check, Quiet and/or VerificationTest?

  • $\begingroup$ Maybe I'll write my own VerificationTest that behaves exactly as I want. $\endgroup$ – masterxilo Sep 1 '16 at 1:54
  • 1
    $\begingroup$ Why would one expect Check to ignore messages just because they were issued within a VerificationTest or why would one expect VerificationTest to detect messages suppressed by Quiet? $\endgroup$ – ilian Sep 1 '16 at 1:57
  • $\begingroup$ @ilian VerificationTest effectively uses something like Quiet to prevent messages generated inside from reaching the front-end. It is confusing that they still reach Check. Check[VerificationTest[Message[a::b]], $Failed] gives $Failed, Check[Quiet[Message[a::b]], $Failed] doesn't, although neither VerificationTest[Message[a::b]] nor Quiet[Message[a::b]] print any messages. $\endgroup$ – masterxilo Sep 1 '16 at 15:10
  • $\begingroup$ @ilian also, usually when Check fails, $MessageList is non-empty, but compare {Check[Message[a::b], $Failed], $MessageList} and {Check[VerificationTest[Message[a::b]], $Failed], $MessageList} $\endgroup$ – masterxilo Sep 1 '16 at 15:12
  • $\begingroup$ (1) VerificationTest does not use Quiet. Still, by design it neither displays messages generated during testing nor puts them on $MessageList (2) Nevertheless, the messages do fire and Check reacts appropriately. For a simplified example, try Check[Block[{$Messages, $MessageList = {}}, 1/0]; {$MessageList, NoMessagesFired}, {$MessageList, SomeMessagesFired}] (3) If you wrapped a Quiet around 1/0 the message would not fire at all. $\endgroup$ – ilian Sep 1 '16 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.