1
$\begingroup$

Is there a way to avoid printing "From KernelObject[n, local]" when Print[] or WriteString[] is used in a parallel computation?

For example,

ParallelDo[WriteString["stdout", i], {i, 1, 4}];

produces

From KernelObject[4, local]:
1
From KernelObject[3, local]:
2
From KernelObject[2, local]:
3
From KernelObject[1, local]:
4

Using Print[i] in place of WriteString["stdout", i] produces the same output (with added newlines). I would like to find a way to have the output be

1
2
3
4

Wrapping either WriteString or ParallelDo with Quiet[] does not seem to have any effect.

I have tried setting Print to Null as suggested in this answer while using WriteString, but "From KernelObject" still appears.

This is for a WolframKernel script, so any expressions that require a frontend cannot be used.

$\endgroup$
  • $\begingroup$ Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Also, please remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! $\endgroup$ – Michael E2 Jun 15 '16 at 19:30
  • 1
    $\begingroup$ Are you okay with not having any control over in what order the results get printed? You might get 1 2 3 4 but you might just as easily get 3 1 4 2. $\endgroup$ – Szabolcs Jun 15 '16 at 20:17
  • $\begingroup$ @Szabolcs Yes, that's fine. $\endgroup$ – reesmichael1 Jun 15 '16 at 21:26
4
$\begingroup$

The simplest and cleanest solution is to use a printing function which evaluates on the main kernel. SetSharedFunction causes a function to always evaluate on the main kernel (despite the somewhat confusing documentation).

Example (run in a terminal):

Mathematica 10.4.1 for Mac OS X x86 (64-bit) Copyright 1988-2016 Wolfram Research, Inc.

In[1]:= print[x___]:=Print[x]                                                                                                               

In[2]:= SetSharedFunction[print]                                                                                                            

In[3]:= ParallelDo[print[i],{i,4}]                                                                                                          
Launching kernels...
1
2
3
4

In general, I recommend to avoid SetSharedFunction because calling back to the main kernel can have a significant performance hit. However, in this case it may not matter. Printing in itself may be nearly as slow anyway (although I did not benchmark this).

Thus in this case I do recommend the above solution.


However, we can try to hack the parallel tools a bit to implement a lower level solution. Just for fun :-) I'll show not only the solution, but also how to arrive at it.

Usual disclaimer: Keep in mind that using undocumented functions and especially modifying builtins is fragile, unsupported and may break your Mathematica session. Proceed with caution!

Unlike most builtin functionality, the parallel tools source code are available. You can find them using

FindFile["Parallel`"]

(* "/Applications/Mathematica 10.4.app/Contents/AddOns/Applications/Parallel/Kernel/init.m" *)

Thus we know to look in AddOns/Applications/Parallel.

The message we want to get rid of is From KernelObject[n, local], where KernelObject[...] could be a separately inserted part. Thus we grep all source files for From.

grep From -R .

This line immediately jumps out:

./Kernels.m:Print[StringForm["From `1`:", kernel]]; Print[text];)

Now we know where to look. Around line 1020 of Kernels.m we find:

(* or print it if no FE *)

printLabelled[text_, kernel_] := (
Print[StringForm["From `1`:", kernel]]; Print[text];)

The simplest hack is to modify this function. Editing source files which are part of Mathematica is just not a good idea. Instead let us patch the function in-memory.

A tricky part in doing this is that we need to know that many definitions are loaded only when first used. Thus before patching the definition, we need to trigger loading it. Usually mentioning just one symbol name from the parlallel tools should suffice.

Demonstration:

enter image description here

Now we are ready to redefine it if we wish:

Parallel`Kernels`Private`printLabelled[text_, kernel_] := ...

But let's not do that. Let us dig further instead. At this point we might have noticed that printLabelled is only called when using the kernel in command line mode. While that is what you want in this case, let us try to understand the printing mechanism used by the parallel tools better.

Aside: At this point it is useful to know what happens when we Print from Mathematica. When the kernel is used in MathLink mode (e.g., it is connected to a front end, or it is connected to another kernel as is the case with parallel kernels), it sends its output as packets. Results may come back as a ReturnPacket (among other types), messages as a MessagePacket and printed stuff as a TextPacket. What the parallel tools do is set up the main kernel to catch any TextPackets and react appropriately. By default this means attaching the kernel labels and re-printing the contents of the packet. Let us continue now.

Looking at the source code might mislead use to think that printLabelled is invoked through printPacket (at least in command line mode). However, following this will lead us into a blind alley, so I'll skip this part.

Looking more carefully at where printLabelled gets called reveals that the crucial function is PacketHandler. We check its context as before:

enter image description here

The relevant definition in the Kernel.m source file is here:

enter image description here

There are different definitions set up for command line mode and for when there is a front end present. We just need to override these definitions. Notice that the definition is attached to TextPacket as an UpValue. This is what we need to change. We must make sure that we use the exact same pattern in our definition as was used originally, so override it. Specifically, notice the part text_String. Using simply text_ here would not override this, but would instead issue a more general definition, and Mathematica would still keep using the original (more specific one). So this is what we need to do:

Mathematica 10.4.1 for Mac OS X x86 (64-bit)
Copyright 1988-2016 Wolfram Research, Inc.

In[1]:= Parallelize                                                             

Out[1]= Parallelize

In[2]:= Unprotect[TextPacket]                                                   

Out[2]= {TextPacket}

In[3]:= TextPacket /:                                                           
         Parallel`Protected`PacketHandler[TextPacket[text_String], kernel_] :=  
          Print["Pack: ", text, " ... ", kernel]                                

In[4]:= ParallelDo[Print[i],{i,4}]                                              
Launching kernels...
Pack: 1 ... KernelObject[4, local]

Pack: 2 ... KernelObject[3, local]

Pack: 3 ... KernelObject[2, local]

Pack: 4 ... KernelObject[1, local]

Now that we understand the packet handling mechanism, we can also override the handler for MessagePacket if we wish.

So that's it. Hope you enjoyed the spelunking tutorial!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.