I have a large $d\times d$ SparseArray
$P$, and I want to multiply a vector (also a SparseArray
) $v$ of length $d$ elementwise with each row of $P$. The value of $d$ is currently $2^{16}=65536$. The matrix $P$ has $d$ entries with the non-default value (which is 1, default is 0). The vector $v$ has $d/2$ non-default entries (again 1's, default 0).
I tried two approaches:
Transpose[v * Transpose[P]]
and
# * v & /@ P
The first version seems to explode in memory, as it immediately brings my computer to a crawl and memory usage goes beyond 90% (i7, 16GB RAM). In the end, the computation officially runs out of memory.
The second version works, but is very slow and uses no more than 4% of my memory. I should note that if I use numerical values in the matrix, the second version seems to parallelize automatically as CPU usage goes to 400%, but it is still very slow. ParallelMap
didn't work apparently, because it is a SparseArray
.
I wonder if there is not a middle way to speed up the computation without blowing my memory?
P.SparseArray[Band[{1, 1}] -> v]
? $\endgroup$