Fast way to pass an array of integers to Java using JLink

Question

I am using a package from java to compress a list of integers, this package provide me two methods, one for compress and other one for uncompress. Each of these methods has a version to int [] arrays or for ArrayList<Integer>. So, the question is about how is the most effective and efficient way to pass an array of integers from mathematica (actually a list like Range[10^6]) to java for these methods, and after that, how can I work with the outcomes after.

I am working with the following approach, pretty slow btw. First, I made a list to test. a2 = Range[1000000];

Needs["JLink`"]
ReinstallJava[];
AddToClassPath[Directory[]];
ShowJavaConsole[]
 LoadJavaClass["example",  StaticsVisible -> False];

AbsoluteTiming[
 JavaBlock@Module[{list},
   list = JavaNew["java.util.ArrayList"]; 
   list@add[MakeJavaObject@#] & /@ a2;
   Print[list@size[]];
   rr = example`basicExample[list];
   ]
 ]

Note 1 : When a2 was till 10000 the process finished after 72. seg. With the version above it seems never ending for a million of integers.

Note 2: For those that they are thinking that it's about examplebasicExample[list];the cause of late, I have to say that with a native array of2500000` integers in java, the program finishes only in 5 segs.

Note 3: I am using a collection ArrayList of Objects because I didn't know how to pass a list of integers in other way, more clever. As you can see, I have to iterate adding each member of a2, indeed, it's awful, at least for me, and the main factor for performance.

Answer 1

The solution

After a standard initialization step

Needs["JLink`"]
InstallJava[]

What you have to do is actually very simple:

arr = MakeJavaObject[{1,2,3}]

(* « JavaObject[[I]» *)

You can test that the object contains your data:

JavaObjectToExpression[arr]

(* {1, 2, 3} *)

Let's test for large array:

(lrg = MakeJavaObject[Range[1000000]])//AbsoluteTiming

(* {0.053362,« JavaObject[[I]»} *)

So, it only takes about 0.05 seconds for a 1000000 elements, which is quite fast. Test:

JavaObjectToExpression[lrg] == Range[1000000]

(* True *)

You can now use this object reference, for example by passing it into other Java functions (methods of some Java classes) from Mathematica - such as e.g. a constructor of some class that would take an integer array as an argument, or any other class method.

Why the slowdown

The reason for dramatic slowdown is that, by appending elements one by one on the Mathematica side, you employ the worst usage pattern possible. The JLink call has a constant overhead, mostly due to lots of symbolic processing steps on the Mathematica side. This overhead can be e.g. 0.001 second, or similar.

In the case when you pass an integer array as a whole, JLink uses optimized ways to do this, and this is vastly faster, because there is just a single JLink function call. In the case of your code, you make as many JLink function calls as there are elements in your array, so this function call overhead totally dominates the running time.

When you use JLink, minimizing a number of JLink function calls and placing your data into efficient data structures (like packed arrays) is critically important if you want decent performance.