# How to speed up summing a large number of vectors

First let me explain what I am trying to do in my code.

Assume that I send a very short chirp (2,000 ms long) into a domain filled with 20,000 particles randomly scattered in the domain.

The backscattered sounds are received by 8 receivers. Because the position of each particle varies in space relative to each receiver, the arrival time of each backscattered chirp varies in time.

Adding up the backscattered chirps from each of the 20,000 particles gives the backscattered signal. The first backscattered chirp arrives at t= 0 ms and the last one at t= 100,000 ms.

In the loop below I want to add up 20,000 signals with length of 100,000. Each of the backscattered signal include only 2,000 non-zero elements and the zeros before and after are added to mark the position of chirp through the time (t= 0 to 100,000).

This process should be repeated for each of the receivers (8 times). I tried to break the 20,000 signals into 20 x 1000 clusters to speed up the process, but still it is very slow.

   bsAll =
ParallelTable[

Block[
{bsi = ConstantArray[0, maxN]},

Do[
Block[{bsCluster, bsClusTot},

bsCluster =
Table[

chirp[[
sstart ;; sfinish[[itr, jj]]]],
ifinish[[itr, jj]]],
maxN]
,
{jj, ii - (cluster - 1), ii}];

],
{ii, cluster, 20000 , cluster}];
bsi
]
,
{itr, ntr}];

sstart = 1
sfinish = is a 8x20000 matrix. most of the components are =2000
ifinish = is a 8x20000 matrix. components are integers between 1 and 98000
maxN = 100000


chirp

https://www.dropbox.com/s/iop6o9k0cr8uzwm/chirp?dl=0

bsAll for cluster=10, and {ii,cluster, 2cluster, cluster}

https://www.dropbox.com/s/nfdembcdtuocxb9/bsAll?dl=0

sfinish:

https://www.dropbox.com/s/xo4k0yz5pozagex/sfinish?dl=0

sstart:

https://www.dropbox.com/s/n9tfqyj9ct5zkjp/sstart?dl=0

ifnish:

https://www.dropbox.com/s/3voa1mo8l24xxan/ifinish?dl=0

• To me, at least, your question is unanswerable without a detailed description of the data you are attempting to process. At a minimum I would like to see a 100 data points that (statistically) model your experimental data. I would also want to see what you would consider a correct summation of the model data. – m_goldberg Sep 14 '15 at 23:24
• the data is the chirp (the first pic, but can be any sinusoidal wave). The final summation is displayed in the second pic. – Mahdi Razaz Sep 14 '15 at 23:46
• Seems to me populating a sparse array directly and just totaling would be vastly more efficient - iterating and doing two paddings each time spells doom for performance. – ciao Sep 14 '15 at 23:56
• I don't have enough RAM to populate the sparse array and then do a total. 20,000 particles are not nearly enough for my work. – Mahdi Razaz Sep 15 '15 at 0:18
• Are the 20,000 signals sorted somehow? – Marius Ladegård Meyer Sep 15 '15 at 8:26