Bug introduced in 8.0 and fixed in 10.0
I'm trying to speed up a tally/binning operation that I have to run many times.
I have many lists of between 1 and 10 numbers from the range [1,5] and I want to Tally
the number of each number and output that in a list.
randomdata = RandomInteger[{1, 5}, #] & /@ RandomInteger[{1, 10}, 100000];
Here is my initial uncompiled implementation using SparseArray
f = Normal@SparseArray[Rule @@@ (Tally@#), 5] &
This works, here is a sample:
f{5, 4, 4, 3, 5, 2, 4, 2, 5, 2}
output: {0, 3, 1, 3, 3}
I'm trying to speed this up with a compiled implementation.
fC = Compile[{{totally, _Integer, 1}},
Module[{tally = Tally[totally], table = Table[0, {i, 5}]},
table[[tally[[All, 1]]]] = tally[[All, 2]];
table],
CompilationTarget -> "C", RuntimeOptions -> "Speed",
RuntimeAttributes -> {Listable}, Parallelization -> True]
This works on the sample
fC[{5, 4, 4, 3, 5, 2, 4, 2, 5, 2}]
output: {0, 3, 1, 3, 3}
but if I do f[randomdata]
the kernel crashes without an error message and I can't figure out what went wrong.
Alternatively, if all the inputs are the same length:
randomdata = RandomInteger[{1, 5}, {100000, 5}];
fC[randomdata]
This works...
It seems that the problem comes down to list over multiple arguments where each one has a different length.
see a simpler example
totalC = Compile[{{list, _Integer, 1}}, Total[list],
CompilationTarget -> "C", RuntimeOptions -> "Speed",
RuntimeAttributes -> {Listable}, Parallelization -> True]
This works: totalC /@ randomdata;
This crashes: totalC[randomdata];
fC/@ randomdata
seems to work, andf[randomdata]
seems also not work on my computer. $\endgroup$