Is there a way to run several RLink computations via REvaluate[] in parallel, e.g. with ParallelTable[]?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ Have you tried your own suggestion? $\endgroup$– Sjoerd C. de VriesCommented Mar 2, 2013 at 9:07
-
$\begingroup$ Yes, but I can't observe a different behavior using Table[] ( comparing the CPU usage and timing). $\endgroup$– RobinautCommented Mar 2, 2013 at 9:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
It is actually possible, if you run several copies of RLink in parallel. This would mean several parallel kernels, R and JVM processes. Here is an example (I have 6 cores):
ParallelEvaluate[Needs["RLink`"]; RLink`InstallR[]]
ParallelEvaluate[RLink`REvaluate["fn <- function(max){sum(sin(1:max))}"]]
and then,
ParallelTable[RLink`RFunction["fn"][i],{i,500000,501000}]//Short//AbsoluteTiming
(* {13.027344,{{1.90482},<<999>>,{1.19949}}} *)
while for a single core, we get:
Needs["RLink`"]
InstallR[]
REvaluate["fn <- function(max){sum(sin(1:max))}"]
and then
Table[RLink`RFunction["fn"][i],{i,500000,501000}]//Short//AbsoluteTiming
(* {57.647461,{{1.90482},<<999>>,{1.19949}}} *)
The more computationally intensive is a single call to R functions, the more you will see the speed benefit of parallelization. Since R is vectorized and also based on immutable expressions, it should be relatively easy to write R functions is a way which would be easy to parallelize and use with parallel Mathematica functionality.
answered Mar 2, 2013 at 15:54
-
$\begingroup$ This question was practically a duplicate of mathematica.stackexchange.com/questions/33/… but my answer there is not great. The general solution is that
Installmust be run separately in each kernel. $\endgroup$– SzabolcsCommented Mar 8, 2013 at 18:44 -
$\begingroup$ @Szabolcs Well, I think it is not quite a duplicate, because here one has to run several JVM and R instances, and while this is possible, it seems good to mention this explicitly. Also, this sort of uses for RLink seem general enough to merit its own discussion, even if this is a special case of the general question you have answered there. $\endgroup$ Commented Mar 8, 2013 at 18:47
-
$\begingroup$ I was not suggesting to close it as a duplicate (I agree it's specific enough), just mentioning that it is the same problem. Mark McClure has an answer about parallel JLink there. $\endgroup$– SzabolcsCommented Mar 8, 2013 at 18:48
-
$\begingroup$ @Szabolcs Actually, this may be important enough to include this as a section into the docs for RLink, for the next release. $\endgroup$ Commented Mar 8, 2013 at 18:48
-
$\begingroup$ @Szabolcs I missed that one. Yes, this is the same idea - R libraries are just loaded into the JVM, so for all practical purposes, this is also just a parallel JLink problem. Actually, at the time, I saw both of your answers and upvoted both, but then I forgot about them. I think, people tend to remember their answers better than the answers of others, alas. $\endgroup$ Commented Mar 8, 2013 at 18:50