5
$\begingroup$

I have an evaluation inside a ParallelTable which sometimes gives me warning messages. After a certain amount of warning messages it will say:

 General::stop :  "Further output of "General::munfl", "MessageName" will be suppressed during this calculation

Unfortunately, this message number limit seems to apply for each parallel kernel separately.

Now, since I calculate on a lot of parallel kernels and the specific message I get is kind of long, I get 10 pages worth of warnings (if I were to print them), so I was wondering whether there is some setting where I can customize how many error messages I want before the General::stop message.

I don't want to completely disable these warnings, which is the only advice I found so far, as it is helpful to know some warnings occured, but as it is now it is very cumbersome to delete all the messages manually each time.

If there is a way to have one kernel print the messages, and mute the others, that would also be a possible solution (At least in my use case, because if one kernel produces error messages, they all do, but this way of solving it might be less useful to other users who search for this problem). Is there a way to do that?

$\endgroup$
4
  • 4
    $\begingroup$ You could redirect your messages to a file instead of the front end - reference.wolfram.com/language/ref/$Messages.html - that way you can still view them but the front end won't become cluttered. $\endgroup$
    – flinty
    Aug 26, 2020 at 13:29
  • $\begingroup$ I don't have time to dig in, but a thought: Quiet[Check[{0^0, 0^0},"ERROR",Power::indet]] suppresses the message, but returns "ERROR" since Check is inside Quiet - this might be a way to generate one message for the whole calculation. $\endgroup$
    – N.J.Evans
    Aug 26, 2020 at 13:30
  • 7
    $\begingroup$ With the menu Mathematica | Preferences ... | Advanced | Open Option Inspector | Global Preferences | Global Options | Message Options | "MaxMessageCount" set value to 1 or whatever value you desire. $\endgroup$
    – Bob Hanlon
    Aug 26, 2020 at 14:21
  • $\begingroup$ Related: mathematica.stackexchange.com/q/58074/280 $\endgroup$ Jan 27, 2022 at 8:06

3 Answers 3

4
$\begingroup$

Let us Trace how messages from parallel kernels are handled in the main kernel:

Flatten[Trace[ParallelTable[Message[General::munfl, $KernelID], {1}], 
    LinkRead | MessagePacket | TextPacket] /. 
   seq : {_HoldForm, _} :> Rule @@ seq] // Column

screenshot

We see that the subkernel first returns MessagePacket[General, "munfl"] which isn't printed, then it returns

TextPacket["General::munfl: 4 is too small to represent as a normalized machine number; precision may be lost."]

which is passed to Parallel`Protected`PacketHandler along with the corresponding KernelObject, from which it extracts the $KernelID and passes it as a part of CellLabel to CellPrint. This is how the original message from the subkernel appears in the evaluation Notebook.

Hence we can intercept the message printing by overloading either Parallel`Protected`PacketHandler or CellPrint. I tried to work with the first without success, but overloading of CellPrint works:

Unprotect[CellPrint];
CellPrint[Cell[s_String, "Print", CellLabel -> label_String, ShowCellLabel -> True]] := 
  Block[{$inCP = True, 
     mname = First[StringCases[s, RegularExpression["^(\\w+::\\w+):"] :> "$1", 1], False]},
     If[! AssociationQ[$SubkernelMessages], $SubkernelMessages = <||>];
     If[! NumericQ[$SubkernelMessagesLimit], $SubkernelMessagesLimit = 3];
     If[KeyExistsQ[$SubkernelMessages, mname], 
       ++$SubkernelMessages[mname], $SubkernelMessages[mname] = 1];
     If[$SubkernelMessages[mname] <= $SubkernelMessagesLimit, 
      CellPrint[Cell[s, "Print", CellLabel -> label, ShowCellLabel -> True]]]; /; 
     StringQ[mname]] /; 
   Not[TrueQ[$inCP]] && StringMatchQ[label, RegularExpression["\\(kernel \\d+\\)"]];
Protect[CellPrint];

After evaluating the above we can use ParallelTable as follows:

$SubkernelMessages =.;
$SubkernelMessagesLimit = 2;
ParallelTable[Message[General::munfl, $KernelID], {100}];

screenshot

Note that $SubkernelMessages is kept during the whole main kernel session, not per evaluation as $MessageList. Hence it should be purged manually before every input where you call a Parallel* function (as I do in the above example) if you wish to mimic the default behavior of Message. However, it is not difficult to automate this too:

Unprotect[CellPrint];
CellPrint[Cell[s_String, "Print", CellLabel -> label_String, ShowCellLabel -> True]] := 
  Block[{$inCP = True, 
     mname = First[StringCases[s, RegularExpression["^(\\w+::\\w+):"] :> "$1", 1], False]},
     If[Not[AssociationQ[$SubkernelMessages]] || $SubkernelMessages["$Line"] =!= $Line, 
       $SubkernelMessages = <|"$Line" -> $Line|>];
     If[! NumericQ[$SubkernelMessagesLimit], $SubkernelMessagesLimit = 3];
     If[KeyExistsQ[$SubkernelMessages, mname], 
       ++$SubkernelMessages[mname], $SubkernelMessages[mname] = 1];
     If[$SubkernelMessages[mname] <= $SubkernelMessagesLimit, 
      CellPrint[Cell[s, "Print", CellLabel -> label, ShowCellLabel -> True]]]; /; 
     StringQ[mname]] /; 
   Not[TrueQ[$inCP]] && StringMatchQ[label, RegularExpression["\\(kernel \\d+\\)"]];
Protect[CellPrint];

With this definition $SubkernelMessagesLimit will limit the number of printed messages from the subkernels for every input $Line of the main kernel:

$SubkernelMessagesLimit = 1;
ParallelTable[Message[General::munfl, $KernelID]; $KernelID, {20}]
ParallelTable[Message[General::munfl, $KernelID]; $KernelID, {20}]

screenshot


As a bonus. If you evaluate (related)

ParallelEvaluate[SetOptions[$Output, PageWidth -> Infinity]];

then the messages from the parallel kernels will be printed without the compulsory linebreaks:

$SubkernelMessages =.;
ParallelTable[Message[General::munfl, $KernelID], {100}];

screenshot

Additionally, you can get a nicer appearance with automating word-wrapping if you change the inner CellPrint part of the above custom definitions to the following:

CellPrint[Cell[BoxData@s, "Message", CellLabel -> label, ShowCellLabel -> True]]

Here is the result:

screenshot

Also, you can get the statistics of the Messages generated in the subkernels:

$SubkernelMessages
<|"General::munfl" -> 36, "General::stop" -> 12|>
$\endgroup$
2
$\begingroup$

If there is a way to have one kernel print the messages, and mute the others, that would also be a possible solution (At least in my use case, because if one kernel produces error messages, they all do, but this way of solving it might be less useful to other users who search for this problem). Is there a way to do that?

Yes, just evaluate Off[General::munfl]; in all parallel kernels excepting one:

LaunchKernels[]

ParallelEvaluate[Off[General::munfl], Rest@Kernels[]]

ParallelTable[Message[General::munfl, $KernelID], {100}];

screenshot


The suggestion to redirect $Messages unfortunately only redirected output from the main kernel, not from the parallel kernels, which is my main problem here.

You need to redefine $Messages in the parallel kernels, not in the main kernel:

ParallelEvaluate[$Messages = {OpenAppend["Messages.txt", PageWidth -> Infinity]}]
{{OutputStream["Messages.txt", 3]}, {OutputStream["Messages.txt", 3]}, {OutputStream["Messages.txt", 3]}, {OutputStream["Messages.txt", 3]}}

Now all the messages from the parallel kernels will be appended to the file "Messages.txt":

ParallelTable[Message[General::munfl, $KernelID], {100}];

Import["Messages.txt"]

screenshot


The suggestion from Bob Hanlon in the comments

With the menu Mathematica | Preferences ... | Advanced | Open Option Inspector | Global Preferences | Global Options | Message Options | "MaxMessageCount" set value to 1

still yielded too much output in my case, but may be the better option for others.

Actually the undocumented "MaxMessageCount" FrontEnd option doesn't affect this behavior at all. Moreover, by default you can't rely on the General::stop when working with subkernels, because General::stop limits the number of messages only for single input (single evaluation), while the Parallel* machinery can break the computation into several evaluations per subkernel:

ParallelTable[$KernelID, {20}]
{4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 4, 4, 3, 3, 2, 2, 1, 1}

You can affect this behavior via Method suboptions though:

ParallelTable[$KernelID, {20}, Method -> "EvaluationsPerKernel" -> 1]
{4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1}

You can overcome this problem without changing the Method suboptions by overloading Message in the subkernels using the technique from this MathGroup post:

ParallelEvaluate[
  $globalMessageList = {};
  Unprotect[Message];
  Message[args___] := 
   Block[{$inMsg = True, $MessageList = $globalMessageList}, Message[args];
     $globalMessageList = $MessageList;] /; ! TrueQ[$inMsg];
  Protect[Message];
  ];

Or better you can use methods from another answer of mine here.

$\endgroup$
0
$\begingroup$

As no one seems interested in answering (in an actual answer), I'll report what I did.

From another question, I got this tip:

ParallelTable[Off[General::munfl];your code here,{your iterator here}]

This turns off all messages unfortunately, but it's the best thing I found in my case.

The suggestion from Bob Hanlon in the comments

With the menu Mathematica | Preferences ... | Advanced | Open Option Inspector | Global Preferences | Global Options | Message Options | "MaxMessageCount" set value to 1

still yielded too much output in my case, but may be the better option for others.

The suggestion to redirect $Messages unfortunately only redirected output from the main kernel, not from the parallel kernels, which is my main problem here.

$\endgroup$
1
  • $\begingroup$ A few days ago I did something like what you seek here, but not in a parallelized evaluation. See the first code block that ends with Tally[Flatten@messages]. $\endgroup$
    – Michael E2
    Jan 2, 2021 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.