12
$\begingroup$

Being able to parallelize Predict /Classify would be useful on multiple fronts:

  • Performance/Scale
  • Streaming
  • Pedagogical
  • Scientific

Performance-wise such parallelization would enable the use of larger datasets and HPC resources; Streaming-wise parallelization would enable certain real-time, distributed, cloud applications; Pedgagogically-wise, parallelization could enhance insights into the structure and judicious choice of predicting/classifying Methods while Scientifically, fine-grained, customized parallelization could promote reproducibility, training variations, extensions of built-in classifiers and the creation of new, hybrid methods.

So are there good frameworks/strategies for parallelizing Predict/Classify?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Just as an observation, more than just parallelization needs to be addressed to make Predict and Classify good vehicles for machine learning research. The black box function approach, whilst good for a fast start, does not lend itself to tinkering, as far too little of the internals of the algorithms are exposed. $\endgroup$
    – image_doctor
    Commented Aug 28, 2015 at 9:19
  • $\begingroup$ @image_doctor It is possible to peer inside PredictorFunction and ClassifierFunction, which may be of help to both combine results of parallel evaluations of Predict and of Classify and to better understand and perhaps modify their numerical parameters. $\endgroup$
    – bbgodfrey
    Commented Aug 28, 2015 at 13:45

1 Answer 1

Reset to default
3
$\begingroup$

Since Version 11.x a new option is available for Predict, Classify, NetTrain: 

TargetDevice -> "GPU"

If you have an NVIDIA graphics card, you can set this to achieve a big speedup in training. Ensure that you have a compatible NVIDIA graphics card and have installed the latest operating system drivers. Works with external GPU too. This guide might come handy for figuring out the GPU connectivity:

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.