5
$\begingroup$

Is it possible to write a LinkedLibrary (or WSTP executable) computing a function f such that

Dynamic@f[]

is updated when the library deems it necessary?

I think it should be possible if the library has a background thread and can somehow change a global symbol within the Mathematica session.

Has anyone ever done this?


Context: I want to replicate

CurrentImage[] // Dynamic

with the output of a simulation or the data read from some unsupported device or something.

Also, I would like to have a variable $file that always reflects the contents of a text file file.txt, even after I change that in an external editor. I know how to listen for changes to a file with the Windows API, so I could do it in C and forward the changes when necessary.

$\endgroup$
  • $\begingroup$ I don't know and I am not sure that you can interact with Dynamic. But let's leave that aside for a moment, and simplify the question: can we have it so that as soon as a file gets modified, the value of $file will change? There's the URLFetchAsynchronous function. When it's done with the download, it can fire off a user specified event. Something like this might work for your purposes. I suspect, but I don't know. that this function uses some of the stuff shown in the async LibraryLink examples. These use functions from some of the ... $\endgroup$ – Szabolcs Feb 22 '17 at 21:38
  • $\begingroup$ ... undocumented LibraryLink headers. I asked about some of them here. I did not have time to look through these examples and understand the functionality they illustrate. If you have time, can you investigate, and report back through a self-answered question? Or at least comment if it can be useful for the question you are asking here? Looking forward to reading about your findings. $\endgroup$ – Szabolcs Feb 22 '17 at 21:40
  • $\begingroup$ SystemOpen@ FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links", "LibraryLink", "LibraryResources", "Source"}] $\endgroup$ – Szabolcs Feb 22 '17 at 21:44
7
$\begingroup$

Preface

I will give a complete solution that shows how a dynamic file watcher can be implemented in Mathematica. The file watcher will track the size of the file and when it changes automatically reload the contents. It will work as an asynchronous library function that does not block the kernel from other evaluations.

The example here will additionally clear most of the issues from the question LibraryLink: Asynchronous Examples. This very same approach can further be used for implementing a CurrentImage[] function.

An AsynchronousTask in Mathematica will have a handler function that is called as soon as the task needs to inform the user about progress. More information about this can be found in the documentation of StartAsynchronousTask. What is still missing in Mathematica is an extensive documentation about how a user can use this in combination with a LibraryLink function. We will see how this can be done and use the handler to receive an event when the file changes, no matter what changed the file. Therefore, you can, for instance, edit the file with a different editor and still get notified inside Mathematica.

Outline of the approach

To make understanding of the C code easier (it's neither complicated nor long), let me outline how the approach works. Will will write a library function that

  1. takes and input filename as a string and a delay between checks in milliseconds.
  2. the library function will create and set up a new asynchronous task that checks the modification time of the file and return this time as an event.

Inside Mathematica, we will start the asynchronous task and use the event inside a Dynamic environment. This ensures that the front end can act upon a change of the modification time of the file.

Implementation

LibraryLink C code

The code can be found here, and I implemented it on Linux. I'm not sure if it works on other operating systems out of the box.

The entry point for us is the function startFileWatcher. This reads the filename and delay arguments and creates a new argument struct that is used to give the asynchronous task all required information. Finally, it creates a new task that will run repeatedly and watches the file.

The function for the task retrieves the provided filename and delay and runs a loop. On each iteration of the loop, the modification time of the file is checked with lstat and this modification time is packed inside a new DataStore. This datastore is what we receive as event in Mathematica when we fire it in line 55

ioLibrary->raiseAsyncEvent(asyncObjID, "change", ds);

As you can see, we can give a string identifier for the event. With more complicated programs this identifier makes an easy pattern matching possible so that we handle different types of events appropriately.

Running in Mathematica

In Mathematica, we first need to compile the library and load the library functions

<< CCompilerDriver`
source = "path/to/file-watcher.c";
lib = CreateLibrary[{source}, "fileWatcherLib"]
startFileWatcher = LibraryFunctionLoad[lib, 
  "startFileWatcher", {Integer, "UTF8String"}, Integer]

Next, we implement a small function the uses this library function and creates an AsynchronousTaskObject from it

StartFileWatcherTask[delayMillis_Integer, fileName_?FileExistsQ, 
  eventHandler_] := 
 Internal`CreateAsynchronousTask[
  startFileWatcher, {delayMillis, fileName}, eventHandler]

The eventHandler function will receive the following arguments when an event is fired

eventHandler[ourAsynchronousTaskObject, eventIdentifierString, dataStore]

In our case, I simply used "change" as identifier here and the datastore will be a list that consists of the modification time as only entry. Therefore, we can define

eventHandler[_, "change", {modTime_}] := (modificationTime = modTime);

and on each event, the global variable modificationTime is updated. Now let us start the task and create a small dynamic part that shows the file contents and updates it on each file change:

task = StartFileWatcherTask[1000, AbsoluteFileName["~/tmp/test.txt"], eventHandler];
Dynamic[modificationTime;
 Module[{txt = Import["~/tmp/test.txt"]},
  Style[txt, Background -> LightBlue
   ]
  ]
 ]

That's it. Now you can edit the test.txt and on save the text in Mathematica updates as well.

Readers that followed me so far will have noticed one thing that shouldn't be done: In the C code loop, I fire an event on each iteration. This is completely unnecesarry. You can simply access the modification time from the former iteration and only fire an event if it changed. With this the communication overhead is reduced to a tiny fraction, especially if you have tasks that are more complex.

$\endgroup$
  • $\begingroup$ But how can you make a change to this variable in response to a file change event that the OS sent, even if an evaluation is not active? $\endgroup$ – Szabolcs Feb 23 '17 at 8:32
  • $\begingroup$ @Szabolcs My idea was to pass the library function a "Shared" tensor event={0}. Inside the library, I can start a new thread that watches the file and increments the tensor on change. The library function returns instantly because the watching happens in a different thread. Every change I make to event there are visible to the kernel. This works well, but unfortunately Dynamic[event] does not work as it seems to be triggered by the MathLink communication between FE and Kernel and not by the value change itself. $\endgroup$ – halirutan Feb 24 '17 at 5:59
  • $\begingroup$ @Szabolcs The better idea would be to use ioLibrary and the async capabilities, but I couldn't find how one actually registers the callback function in case of an event. There are 3 or 4 examples and they left out this tiny detail. I would assume that this can be used with AsynchronousTaskObject but who knows, when the whole documentation just concerns fetching an URL :( $\endgroup$ – halirutan Feb 24 '17 at 6:12
  • 1
    $\begingroup$ @Szabolcs I found how to use LibraryLinke with and asynchronous task. So I guess if I don't want to delete this answer I have to code a small example. $\endgroup$ – halirutan Feb 24 '17 at 6:17
  • $\begingroup$ That's exactly the problem: if a shared tensor is modified from a different thread, then the kernel is not notified of this modification and cannot do whatever it usually does to trigger Dynamic stuff (I assume send signals to the FE through MathLink, as you said). $\endgroup$ – Szabolcs Feb 24 '17 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.