I have gone through the reference titled Advanced Manipulate Functionality, and I quote from the section titled Dealing with Slow Evaluations
:
There is no five-second limit to such nonblocking evaluations, so by using the ContinuousAction -> False option, arbitrarily long evaluations can be used.
This appears to be incorrect. Even with ContinuousAction -> False
my Manipulate
evaluation is aborting after a few seconds. My only recourse is to pre-compute everything, but this consumes a lot of memory.
Unfortunately, I cannot provide a direct sample of the code, because it involves loading large CSV files residing on my disk. Each selection of Manipulate
should have loaded all the files in the selected directory and done some computations on them. Instead, I have to cycle through the directories and load all the files first, and then call the loaded lists in Manipulate
. An individual file takes about 10 to 15 seconds to load, but the entire collection takes over 20 minutes. It would have been better if I could have loaded the files on demand.
Are there any other hidden options for Manipulate
that I might use here?
EDIT
In response to @Edmund's request, I am providing a pared-down version of the code. First, the helper functions:
ClearAll[customManipulate, nestedManipulate, loadCSVFiles,];
customManipulate[cf_, lbl_] := With[{d = #, k = Sort@Keys@#}, Manipulate[
cf[d[s]], {{s, First@k, Style[lbl, "Subsubsection"]}, k},
Paneled -> False, FrameMargins -> None, ControlType -> PopupMenu,
ContinuousAction -> False
]] &;
nestedManipulate[ifn_, lbls_] := Fold[customManipulate, ifn, Reverse@lbls];
loadCSVFiles = RightComposition[
FileNames["*.csv", #] &, AssociationMap[Import[#, "CSV"] &], KeyMap[FileBaseName]
];
The function nestedManipulate
is simply a convenience function that allows me to call customManipulate
on a deeply nested Association
.
Now, say that I have a folder called csv-dat
, which contains CSV files, and a data-processing function dfn
. I can use the following lines of code to view the files:
{"path", "to", "csv-dat"} // FileNameJoin // loadCSVFiles // customManipulate[dfn, "File"]
This should give me a Manipulate
view from where I can select the particular file whose result I want to view.
I use this setup quite often in internal demonstrations. Let us say that the folder csv-dat
has about 600 files, and the function dfn
takes about 10 seconds to run on each file. During the demonstration, I don't mind waiting for 10 seconds each time, especially since I can talk about other aspects of the data in the meantime. But I don't want to wait for over 6,000 seconds to preprocess every file before starting the meeting. Nor can I necessarily preprocess only a small subset, because at various times, different teams want to view different files (usually based on their names).
The other thing is that often I use more than one data processing function. And I want to be able to select between dfn1
and dfn2
without needing to preprocess everything.
So, this is the predicament I face. And I would be grateful if someone could help me find a set-up so that customManipulate
did not abort.
Pause
in place of the actual operation that takes a long time. It is not possible to help you without a minimal working example of the issue. In doing this you may actually find what is odd in the code or the behaviour. $\endgroup$