7
$\begingroup$

I have a function the takes some time to complete. The function imports data from a .csv, parses it, and plots certain vectors from there. The .csv is constantly being updated from an external program. Ideally, the file size of the .csv would be monitored and when it changes the function is run. Here's what I have so far (assuming the described function is f).

file = "testfile.csv"
Dynamic[
    Refresh[
     filesize:=FileByteCount[file];
     data = Import[file, "CSV"];
     f[data],TrackedSymbols:>{filesize},UpdateInterval->5]]

This works, but seems to run f every 5 seconds no matter if the file size has changed or not. This doesn't seem to be very efficient. Also, if I try to use inside DynamicModule it doesn't work at all.

So, what's the best way to monitor and execute code on a file change?

|improve this question
$\endgroup$
9
$\begingroup$

The best way to do this would be using scheduled tasks. This way, you have greater flexibility over starting/stopping and quitting the task than with Dynamic. Here's an example:

Begin["Kale`"];
    fileName = "~/foo.txt";
    lastModified = {};
    updatedQ := With[{modificationDate = FileDate[fileName, "Modification"]}, 
        If[lastModified == modificationDate, False, lastModified = modificationDate; True]];
    task = CreateScheduledTask[If[updatedQ, Print["Changed"], ## &[]], {2, ∞}];
End[];

The above will check every 2 seconds to see if your file has been modified. If it has been, then it prints "Changed" (replace it with your custom function) to the messages window and does nothing otherwise.

You can start and stop your task with the following commands

StartScheduledTask[Kale`task];
StopScheduledTask[Kale`task];

respectively or remove it altogether using RemoveScheduledTask[Kale`task];.

|improve this answer
$\endgroup$
  • $\begingroup$ @rm -rf, So, I'm gathering the best way to dynamically plot the data is to use the scheduled task to format the vectors for plotting, set to a variable, and then use Dynamic[Plot[stuffs]]? $\endgroup$ – kale Dec 7 '12 at 20:13
  • $\begingroup$ @kale Yes, that would seem like a logical way to do it. It works very well in my example tests. However, you should also take your problem's constraints and size into account. For instance, reading in a huge data file every few seconds would be insane, so you might have to cache the results and read incrementally, etc. $\endgroup$ – rm -rf♦ Dec 7 '12 at 20:44
  • $\begingroup$ @kale Depending on the type of file that's being updated, FileHash[] might be a worthy alternative. (I don't know how much latency is involved with operating systems updating time stamps of particular files, but would imagine it could vary based on type of file, size of file, and how it's being changed---is data 'streaming' into it?.) $\endgroup$ – telefunkenvf14 Dec 8 '12 at 5:33
  • $\begingroup$ @telefunkenvf14, Yeah, data is streaming into it from that aforementioned external program. I agree on the time stamp thing, so I am still relying on FileByteCount. FileHash could be faster though... $\endgroup$ – kale Dec 8 '12 at 14:55
  • $\begingroup$ @telefunkenvf14, I just checked. FileByteSize is much much faster. $\endgroup$ – kale Dec 8 '12 at 16:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.