I am interested in calculating the cosine distance between each pair of the element of a sparse matrix. I am using the built-in function DistanceMatrix
with the option CosineDistance
.
My data is a sparse matrix sp
with dimension ~{30000,6}
and number of non zero vectors in sp
is ~3000. I calculate the distance matrix in the following manner:
sp = SparseArray[
Table[{RandomInteger[{1, 30000}], RandomInteger[{1, 6}]} ->
RandomReal[1], {3000}]]
DistanceMatrix[sp,
DistanceFunction -> CosineDistance]; // AbsoluteTiming
The calculation does not converge, any suggestion why the calculation does not converge.
I have a list of sparse matrices, and for each one of them, I want to calculate the cosine distance. I calculate the distance matrix with @Henrik
solution.
When I run the code outside Module (or Table
as in the following example), I get the distance matrix with right dimensions, but when I run it inside Module
, I get wrong dimensions of the distance matrix.
SeedRandom[123];
sp = matrices[[1]];
ilist = Flatten[
SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]];
B = SparseArray[{}, {1, 1} Length[sp], 0.];
B[[ilist, ilist]] =
SparseArray@
DistanceMatrix[sp[[ilist]],
DistanceFunction -> CosineDistance]; // AbsoluteTiming // First
Table[
sp = matrices[[i]];
ilist = Flatten[
SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]];
BB = SparseArray[{}, {1, 1} Length[sp], 0.];
BB[[ilist, ilist]] =
SparseArray@
DistanceMatrix[sp[[ilist]],
DistanceFunction -> CosineDistance], {i, 1}]
Table[sp = matrices[[i]]; ilist = Flatten[ SparseArray[Unitize[Total[Abs[sp], {2}]]]["NonzeroPositions"]]; BB = SparseArray[{}, {1, 1} Length[sp], 0.]; BB[[ilist, ilist]] = SparseArray@ DistanceMatrix[sp[[ilist]], DistanceFunction -> CosineDistance]; BB, {i, 1} ]
. The return value should beBB
. But the return value ofBB[[ilist, ilist]] = ...
is only the submatrix. $\endgroup$