12
$\begingroup$

I have a function the takes some time to complete. The function imports data from a .csv, parses it, and plots certain vectors from there. The .csv is constantly being updated from an external program. Ideally, the file size of the .csv would be monitored and when it changes the function is run. Here's what I have so far (assuming the described function is f).

file = "testfile.csv"
Dynamic[
    Refresh[
     filesize:=FileByteCount[file];
     data = Import[file, "CSV"];
     f[data],TrackedSymbols:>{filesize},UpdateInterval->5]]

This works, but seems to run f every 5 seconds no matter if the file size has changed or not. This doesn't seem to be very efficient. Also, if I try to use inside DynamicModule it doesn't work at all.

So, what's the best way to monitor and execute code on a file change?

$\endgroup$
0

2 Answers 2

13
$\begingroup$

The best way to do this would be using scheduled tasks. This way, you have greater flexibility over starting/stopping and quitting the task than with Dynamic. Here's an example:

Begin["Kale`"];
    fileName = "~/foo.txt";
    lastModified = {};
    updatedQ := With[{modificationDate = FileDate[fileName, "Modification"]}, 
        If[lastModified == modificationDate, False, lastModified = modificationDate; True]];
    task = CreateScheduledTask[If[updatedQ, Print["Changed"], ## &[]], {2, ∞}];
End[];

The above will check every 2 seconds to see if your file has been modified. If it has been, then it prints "Changed" (replace it with your custom function) to the messages window and does nothing otherwise.

You can start and stop your task with the following commands

StartScheduledTask[Kale`task];
StopScheduledTask[Kale`task];

respectively or remove it altogether using RemoveScheduledTask[Kale`task];.

$\endgroup$
7
  • $\begingroup$ @rm -rf, So, I'm gathering the best way to dynamically plot the data is to use the scheduled task to format the vectors for plotting, set to a variable, and then use Dynamic[Plot[stuffs]]? $\endgroup$
    – kale
    Dec 7, 2012 at 20:13
  • $\begingroup$ @kale Yes, that would seem like a logical way to do it. It works very well in my example tests. However, you should also take your problem's constraints and size into account. For instance, reading in a huge data file every few seconds would be insane, so you might have to cache the results and read incrementally, etc. $\endgroup$
    – rm -rf
    Dec 7, 2012 at 20:44
  • $\begingroup$ @kale Depending on the type of file that's being updated, FileHash[] might be a worthy alternative. (I don't know how much latency is involved with operating systems updating time stamps of particular files, but would imagine it could vary based on type of file, size of file, and how it's being changed---is data 'streaming' into it?.) $\endgroup$ Dec 8, 2012 at 5:33
  • 1
    $\begingroup$ @R.M. I have tried your code on version 11.0.1.0 seems not working anymore. Could you check please? $\endgroup$
    – Leo1215
    Apr 6, 2017 at 23:51
  • 1
    $\begingroup$ It works in 12.0 if you replace lastModified = {}; with lastModified =FileDate[fileName, "Modification"] $\endgroup$
    – Conor
    Feb 12, 2021 at 20:54
6
$\begingroup$

rm -rf's answer (above) works in Mathematica 12.0 if you replace lastModified = {}; with lastModified =FileDate[fileName, "Modification"]


Dynamic[x]

.

Begin["Kale`"];

myCode[data_] := Style[ListLinePlot[Flatten[data], PlotTheme -> "Marketing"], Magnification -> 2];
fileName = NotebookDirectory[] <> "file.txt";

lastModified = FileDate[fileName, "Modification"];

updatedQ := With[{modificationDate = FileDate[fileName,"Modification"]}, 
   If[lastModified == modificationDate, False, 
    lastModified = modificationDate; True]];

task = CreateScheduledTask[
                 If[
                    updatedQ
    , (
     data = Import[fileName, "Table"]; x = myCode[data]
      )
    , ## &[]
                  ]
                  , {0.1, ∞}
             ];

End[];

.

StartScheduledTask[Kale`task]

enter image description here


note:

in this example, file.txt was created with:

Export[NotebookDirectory[]<>"file.txt",{1,2,3}]//SystemOpen
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.