Can you trigger update of Dynamic elements from a LinkedLibrary?

Question

Is it possible to write a LinkedLibrary (or WSTP executable) computing a function f such that

Dynamic@f[]

is updated when the library deems it necessary?

I think it should be possible if the library has a background thread and can somehow change a global symbol within the Mathematica session.

Has anyone ever done this?

---

Context: I want to replicate

CurrentImage[] // Dynamic

with the output of a simulation or the data read from some unsupported device or something.

Also, I would like to have a variable $file that always reflects the contents of a text file file.txt, even after I change that in an external editor. I know how to listen for changes to a file with the Windows API, so I could do it in C and forward the changes when necessary.

Answer 1

Preface

I will give a complete solution that shows how a dynamic file watcher can be implemented in Mathematica. The file watcher will track the size of the file and when it changes automatically reload the contents. It will work as an asynchronous library function that does not block the kernel from other evaluations.

The example here will additionally clear most of the issues from the question LibraryLink: Asynchronous Examples. This very same approach can further be used for implementing a CurrentImage[] function.

An AsynchronousTask in Mathematica will have a handler function that is called as soon as the task needs to inform the user about progress. More information about this can be found in the documentation of StartAsynchronousTask. What is still missing in Mathematica is an extensive documentation about how a user can use this in combination with a LibraryLink function. We will see how this can be done and use the handler to receive an event when the file changes, no matter what changed the file. Therefore, you can, for instance, edit the file with a different editor and still get notified inside Mathematica.

Outline of the approach

To make understanding of the C code easier (it's neither complicated nor long), let me outline how the approach works. Will will write a library function that

1. takes and input filename as a string and a delay between checks in milliseconds.
2. the library function will create and set up a new asynchronous task that checks the modification time of the file and return this time as an event.

Inside Mathematica, we will start the asynchronous task and use the event inside a Dynamic environment. This ensures that the front end can act upon a change of the modification time of the file.

Implementation

LibraryLink C code

The code can be found here, and I implemented it on Linux. I'm not sure if it works on other operating systems out of the box.

The entry point for us is the function startFileWatcher. This reads the filename and delay arguments and creates a new argument struct that is used to give the asynchronous task all required information. Finally, it creates a new task that will run repeatedly and watches the file.

The function for the task retrieves the provided filename and delay and runs a loop. On each iteration of the loop, the modification time of the file is checked with lstat and this modification time is packed inside a new DataStore. This datastore is what we receive as event in Mathematica when we fire it in line 55

ioLibrary->raiseAsyncEvent(asyncObjID, "change", ds);

As you can see, we can give a string identifier for the event. With more complicated programs this identifier makes an easy pattern matching possible so that we handle different types of events appropriately.

Running in Mathematica

In Mathematica, we first need to compile the library and load the library functions

<< CCompilerDriver`
source = "path/to/file-watcher.c";
lib = CreateLibrary[{source}, "fileWatcherLib"]
startFileWatcher = LibraryFunctionLoad[lib, 
  "startFileWatcher", {Integer, "UTF8String"}, Integer]

Next, we implement a small function the uses this library function and creates an AsynchronousTaskObject from it

StartFileWatcherTask[delayMillis_Integer, fileName_?FileExistsQ, 
  eventHandler_] := 
 Internal`CreateAsynchronousTask[
  startFileWatcher, {delayMillis, fileName}, eventHandler]

The eventHandler function will receive the following arguments when an event is fired

eventHandler[ourAsynchronousTaskObject, eventIdentifierString, dataStore]

In our case, I simply used "change" as identifier here and the datastore will be a list that consists of the modification time as only entry. Therefore, we can define

eventHandler[_, "change", {modTime_}] := (modificationTime = modTime);

and on each event, the global variable modificationTime is updated. Now let us start the task and create a small dynamic part that shows the file contents and updates it on each file change:

task = StartFileWatcherTask[1000, AbsoluteFileName["~/tmp/test.txt"], eventHandler];
Dynamic[modificationTime;
 Module[{txt = Import["~/tmp/test.txt"]},
  Style[txt, Background -> LightBlue
   ]
  ]
 ]

That's it. Now you can edit the test.txt and on save the text in Mathematica updates as well.

Readers that followed me so far will have noticed one thing that shouldn't be done: In the C code loop, I fire an event on each iteration. This is completely unnecesarry. You can simply access the modification time from the former iteration and only fire an event if it changed. With this the communication overhead is reduced to a tiny fraction, especially if you have tasks that are more complex.