I'm currently doing work on processing a number of images taken frame-by-frame from a video. As a result, I have a directory of around 16K PNG images that at most ~300K.
That said, I have a routine,
analyze which I map in the following way:
analyse /@ FileNames[ "captures\\*.png" , nbDir];
- Possibly some mask operations using
- Assembling the images using
That said, I'm heavily computationally bound for the the process I perform on each image, but also heavily disk/IO bound because there are so many images.
To that end, I've started using
ParallelMap like so:
ParallelMap[analyse, FileNames["captures\\*.png" , nbDir]];
However, I know of the
Method parameter which allows one to specify just how much parallelization should take place in
My gut instinct here is to go with the
CoarsestGrained option for the
Method parameter but I can't really put together why the approach would be valid (or invalid, if it were not).
For this situation, what approach can I take to tune this call to
ParallelMap in order to get the best performance (to be specific, I'd like it to be done as soon as possible, with a disregard for resource consumption if sacrifices are to be made).