Cosine similarity calculation

Question

I am interested in calculating the cosine distance between each pair of the element of a sparse matrix. I am using the built-in function DistanceMatrix with the option CosineDistance.

My data is a sparse matrix sp with dimension ~{30000,6} and number of non zero vectors in sp is ~3000. I calculate the distance matrix in the following manner:

sp = SparseArray[
  Table[{RandomInteger[{1, 30000}], RandomInteger[{1, 6}]} -> 
    RandomReal[1], {3000}]]

 DistanceMatrix[sp, 
    DistanceFunction -> CosineDistance]; // AbsoluteTiming

The calculation does not converge, any suggestion why the calculation does not converge.

I have a list of sparse matrices, and for each one of them, I want to calculate the cosine distance. I calculate the distance matrix with @Henrik solution. When I run the code outside Module (or Table as in the following example), I get the distance matrix with right dimensions, but when I run it inside Module, I get wrong dimensions of the distance matrix.

SeedRandom[123];
sp = matrices[[1]];
ilist = Flatten[
   SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]];
B = SparseArray[{}, {1, 1} Length[sp], 0.];
B[[ilist, ilist]] = 
    SparseArray@
     DistanceMatrix[sp[[ilist]], 
      DistanceFunction -> CosineDistance]; // AbsoluteTiming // First

Table[
 sp = matrices[[i]];
 ilist = Flatten[
   SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]];
 BB = SparseArray[{}, {1, 1} Length[sp], 0.];
 BB[[ilist, ilist]] = 
  SparseArray@
   DistanceMatrix[sp[[ilist]], 
    DistanceFunction -> CosineDistance], {i, 1}]

Answer 1

You can try the following. It computes the distance matrix of merely the nonzero rows.

SeedRandom[123];
sp = SparseArray[ Table[{RandomInteger[{1, 30000}], RandomInteger[{1, 6}]} -> RandomReal[1], {3000}]];
ilist = Flatten[SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]];
A = SparseArray[{}, {1, 1} Length[sp], 0.];
A[[ilist, ilist]] = SparseArray@DistanceMatrix[
 sp[[ilist]], 
 DistanceFunction -> CosineDistance
 ]; // AbsoluteTiming // First

0.384662