# local logfiles for remote kernels

I'd like remote slave kernels to produce log files on my local machine, subject to the following constraints:

1) I don't want to install a distributed filesystem or do other administrative work on the remote machines. I am not their administrator.

2) I don't want to rewrite the interaction between master/slave kernels to support explicit passing of log messages. The slaves are doing long-running computations with lots of state; they need time to finish. On the other hand I need to see the log files as the computations are running.

I know that it is possible to intercept a message (see here) and run a snippet of code, without disturbing the main line of computation (The answer in the link aborts the computation, but you could just as easily do something less intrusive). Is there some way, spiritually along these lines, to generate and then intercept an interprocess communication, which I can use to cause the master kernel to write log entries to files on my local machine?

-
It's possible to make certain functions always evaluate on the main kernel, even if they're called on a parallel kernel. Would this solution be acceptable for you? It'll have some overhead (I believe the parallel kernel will have to wait until the evaluation finishes on the main kernel, so you have to be careful what you use this for), but for logging it should not be a problem. –  Szabolcs Jan 22 '13 at 21:50
@Szabolcs I'm delighted to learn about SetSharedFunction. My main kernel is running a WaitNext loop. I'm traversing a tree, the structure of which I do not know in advance, so I schedule nodes for processing via ParallelSubmit as I discover them. It sounds like you are saying that SetSharedFunction won't work in my situation. –  Tobias Hagge Jan 22 '13 at 22:49
@Szabolcs I think I did not understand your comment about waiting until evaluation finishes. I just checked and WaitNext does not block shared functions from evaluating, so setSharedFunction should work for me. I will verify that that it does and accept your answer below. –  Tobias Hagge Jan 22 '13 at 23:18
Sorry for being unclear, I believe SetSharedFunction should work in your case. What I meant is that it's not a good idea to use it on functions that take a long time to finish, but it should be excellent for logging purposes. –  Szabolcs Jan 22 '13 at 23:44

One way would be to just use Print. Whenever a subkernel prints something, it is immediately sent to the main kernel and ultimately displayed in the notebook, like any other print message.

If this is too simple, you can create your own logging function and set it to be evaluated on the main kernel using SetSharedFunction. (This is not clearly documented, but what SetSharedFunction does is makes a function always run on the main kernel.)

Here is an example demonstrating both possibilities. For illustration, I'm running two remote subkernels ("tosh") and 8 local ones. Let's define a "logging" function that returns the ID of the kernel it ran on:

log[x_] := (Print[x]; \$KernelID)


Then let's try to use it in a parallel calculation first in the usual way, then after calling SetSharedFunction on it. I included the results as a screenshot to clearly show when some messages are coming from subkernels.

-
Thanks. This is exactly what I asked for. For others reading this question I'll mention that there's a noticable performance hit (as measured by CPU usage on the parallel kernels) if more than a few calls per second to a shared function are made, even in a minimal example, so for verbosely logged programs it's better just to log to the remote machines if it's an option. –  Tobias Hagge Jan 24 '13 at 5:20