# Parallelizing sparse array construction

I have a very large sparse array, and the computation of individual matrix elements is fairly expensive. The Matrix is Hermitian and traceless, so I would like to construct only the sub-diagonal elements explicitly.

A sketch of my attempt:

M = SparseArray[{}, {imax, imax}];
SetSharedVariable[M];
ParallelDo[
If[j < i, M[[i, j]] = f[i, j]],
{i, 1, imax}, {j, 1, imax}];


My understanding is that setting M as a shared variable this way is very expensive. Is there a good way to parallelize this process?

Note: I have seen examples where people calculate dense matrices by constructing the submatrices on separate kernels, but for my matrix the computation time of a submatrix is difficult to estimate so trying to distribute the computation time manually is difficult.

-
Try to use the common scheme: SparseArray[ids -> vals], where ids = {{i1,j1},{i2,j2},...} is a list of indexes and vals is a list of correcpoding values. You can calulate vals with ParallelTable. Moreover, adding elements to existing SparseArray takes sensible amount of time. – ybeltukov Sep 1 '14 at 19:07

## 1 Answer

You can use ParallelTable and generate the SparseArray form the table.

f[i_, j_] := i + j;
imax = 50;

AbsoluteTiming[
M = SparseArray[{}, {imax, imax}];
SetSharedVariable[M];
ParallelDo[If[j < i, M[[i, j]] = f[i, j]], {i, 1, imax}, {j, 1, imax}]
]
(*{8.259472, Null}*)

AbsoluteTiming[
M2 = SparseArray[Flatten[ParallelTable[{{i, j} -> f[i, j]}, {i, 1, imax}, {j, 1, i - 1}]],{imax,imax}];
]
(*{0.038002,Null}*)

-
Why the j=2? It seems to work without that. – Cory Schillaci Sep 1 '14 at 19:35
Ups, I forgot to remove this from the code. Corrected, Thanks. – paw Sep 1 '14 at 19:43
Also the sparse array shape should be specified, otherwise it's not a square matrix. Thanks for you answer! – Cory Schillaci Sep 1 '14 at 19:44
Good point, if your matrix has to be square you can specifiy the size as you did in your question. – paw Sep 1 '14 at 19:53