Speeding up Position for finding the indices of a small number of integer elements in a large matrix

Question

I have a large $100 \times 4000$ element matrix composed of integer elements, testMatrix, and I was to use Position to find the indices of integers larger than some threshold value thresholdValue. The following works:

Position[testMatrix, x_ /; x >= thresholdValue, 2]

However it takes over $\approx 150$ milliseconds to execute.

Given that the number of positions is typically around $10^3$, is there a way to achieve a significant speedup for this procedure?

Answer 1

SeedRandom[1];
testMatrix = RandomInteger[10^4, {100, 4000}];
Position[testMatrix, x_ /; x >= 1000] // Hash // AbsoluteTiming
Position[UnitStep[testMatrix - 1000], 1] // Hash // AbsoluteTiming
Compile[{}, Position[UnitStep[testMatrix - 1000], 1]][] //  Hash // AbsoluteTiming
SparseArray[UnitStep[testMatrix - 1000]]["NonzeroPositions"] // Hash // AbsoluteTiming

(*
{0.665038, 953655867}
{0.282016, 953655867}
{0.038002, 953655867}
{0.032002, 953655867}
*)

Answer 2

With compile little more speed up!

f = Compile[{{mat, _Integer, 2}, {th, _Real}}, 
     Position[(Map[Boole[# > th] &, mat, {2}]), 1], 
     CompilationTarget -> "C", Parallelization -> True, 
     RuntimeOptions -> "Speed", RuntimeAttributes -> {Listable}];

Test

SeedRandom[1];
mat = RandomInteger[10^4, {100, 4000}];
f[mat, 1000]; // AbsoluteTiming
Position[mat, x_ /; x >= 1000]; // AbsoluteTiming
Position[UnitStep[mat - 1000], 1]; // AbsoluteTiming
SparseArray[UnitStep[mat - 1000]]["NonzeroPositions"]; // AbsoluteTiming

{0.015014, Null}

{0.398360, Null}

{0.293312, Null}

{0.032029, Null}