A fast way to compute arrays avoiding If or Which

I found out that if I have to calculate huge arrays with most of the elements being zero( let's say each row has 10000 elements and only 8 are non zero) and the non zero elements' positions being given by an If or a Which, the most straight forward way to go is:

Table[...,{i,1,nn},{j,list[i]}]


Table[If[MemberQ[list[i],j]==True,...],{i,1,nn},{j,1,nn}]


But the problem is that, list[i] does not fix the position in the array, but only the value, the positions being [1,Length[list[i]]]. Let us ilustrate this with an easy example:

Slow way:

A = Table[
Which[MemberQ[{1, 3, 5}, j] == True, 2*i + j, True, 0], {i, 1, 2}, {j,
5}] // MatrixForm
(*(3    0   5   0   7
5    0   7   0   9)*)


Fast way:

B=Table[2*i + j, {i, 1, 2}, {j, {1, 3, 5}}] // MatrixForm

(*(3    5   7
5    7   9)*)


So, I would like to get the matrix A but doing it B way somehow.

(The condition for j, is an example, but in the real code is a rather complicated condition that varies with i.)

Does anybody see how this could be done?

Thanks

• Have you considered SparseArray? – xzczd Dec 17 '13 at 11:23
• @xzczd yeah I tried many many things, as far as I am aware, you can not do this with SparseArrays, the most similar you can do with SparseArray is using Condition with SparseArrays, but that is slow. – Mencia Dec 17 '13 at 11:27
• By the way you never have to write x == True ;) instead you can write x – Jacob Akkerboom Dec 17 '13 at 17:13

As a complement to Anon's answer you can use pattern based rules in SparseArray, but depending on the pattern it may not be fast. (It will still have the memory advantage of a sparse array). For example:

SparseArray[{i_, j:(1|3|5)} :> 2*i + j, {2, 5}] // MatrixForm


$\left( \begin{array}{ccccc} 3 & 0 & 5 & 0 & 7 \\ 5 & 0 & 7 & 0 & 9 \end{array} \right)$

Alright, so your $j$ indexes are dependent on a function of the $i$ index. You could do this with Condition as follows, but performance will be poor. At the very least memoize the index function:

f[n_] := f[n] = Mod[Multinomial[Range@3, n], 5, 1]

SparseArray[{i_, j_} /; MemberQ[f@i, j] :> 2*i + j, {3, 5}] // MatrixForm


$\left( \begin{array}{ccccc} 0 & 4 & 5 & 6 & 0 \\ 5 & 0 & 7 & 0 & 9 \\ 0 & 0 & 0 & 10 & 11 \end{array} \right)$

It would be better to build a position list as Anon described in a comment. Here is a more complete example:

SparseArray[
{##} -> 2 # + #2 & @@@ Join @@ Array[Thread@{#, f@#} &, 3]
] // MatrixForm


$\left( \begin{array}{ccccc} 0 & 4 & 5 & 6 & 0 \\ 5 & 0 & 7 & 0 & 9 \\ 0 & 0 & 0 & 10 & 11 \end{array} \right)$

In a somewhat different style for use with listable inner functions (the 2*i + j part):

is = Join @@ (ConstantArray[Range@3, 3]\[Transpose])
js = Join @@ Array[f, 3]

SparseArray[Thread[{is, js}] -> 2 is + js] // MatrixForm

{1, 1, 1, 2, 2, 2, 3, 3, 3}

{2, 3, 4, 3, 1, 5, 4, 5, 5}


$\left( \begin{array}{ccccc} 0 & 4 & 5 & 6 & 0 \\ 5 & 0 & 7 & 0 & 9 \\ 0 & 0 & 0 & 10 & 11 \end{array} \right)$

• thanks. Let us say we have the following function that generates the previously mentioned list: Nx = 5; Ny = 5; N1[i_] := Module[{ix, iy, n1x, n2x, n3y, n4y}, ix = Mod[i, Nx, 1]; iy = Quotient[i, Nx, 1] + 1; If[ix + 1 > Nx, n1x = 1, n1x = ix + 1]; If[ix - 1 < 1, n2x = Nx, n2x = ix - 1]; If[iy + 1 > Ny, n3y = 1, n3y = iy + 1]; If[iy - 1 < 1, n4y = Ny, n4y = iy - 1]; {n1x + (iy - 1)*Nx, n2x + (iy - 1)*Nx, ix + (n3y - 1)*Nx, ix + (n4y - 1)*Nx}]; – Mencia Dec 17 '13 at 12:28
• Then, something similar to what you suggested but that is is not allowed would be: SparseArray[{i_, j : (N1[i_][[1]] | N1[i_][[2]] | N1[i_][[3]] | N1[i_][[4]])} :> 2*i + j, {NxNy, NxNy}] // MatrixForm – Mencia Dec 17 '13 at 12:32
• So the point is, the condition for which j to choose, is given by function N1[i] – Mencia Dec 17 '13 at 12:33
• Could you do it using N1? – Mencia Dec 17 '13 at 12:58
• @Mencia OK, I see your problem. Give me a few minutes to update my answer. – Mr.Wizard Dec 17 '13 at 13:28

In example B you know the different indices, i.e. you know the positions of the elements. If you know the positions of the elements, you can use SparseArray:

SparseArray[Rule[#, 2 #[[1]] + #[[2]]] & /@ Tuples[{{1, 2}, {1, 3, 5}}]] // MatrixForm
(*(3    0   5   0   7
5    0   7   0   9)*)

• thanks so much for the answer but, it does not really work for me because in my code it goes like ...{i,1,nn},{j,list[i]}], the list is i dependent, so it would be nice to have index i somewhere – Mencia Dec 17 '13 at 11:34
• @Mencia OK, you need a different code to generate your positions but the principle is still valid. You have all the information you need to calculate the positions. Perhaps something like Flatten[Thread[{list[#], #}] & /@ Range[nn], 1]. – C. E. Dec 17 '13 at 11:48

If you want to create a PackedArray, rather than a SparseArray object, the following is an option

cfu =
Compile[
{{list, _Integer, 1}, {nn, _Integer}}
,
Block[
{max = Max[list], len = Length[list], res}
,
res = ConstantArray[0, {nn, max}];
Do[
If[
list[[ll]] == j
,
res[[i, j]] = 2*i + j
]
,
{i, 1, nn},
{j, 1, max},
{ll, 1, len}
];
res
]
]


example

cfu[{1, 3, 5}, 2] // MatrixForm
(*output*)
(*(3    0   5   0   7
5    0   7   0   9)*)


The result is indeed a PackedArray

<<Developer
PackedArrayQ@cfu[Range[1000], 3]


True

• Oh I should probably add CompilationTarget-> "C". I hope I can compare timings later – Jacob Akkerboom Dec 17 '13 at 15:39
• I await your timings. – Mr.Wizard Dec 17 '13 at 15:42

Maybe I'm just simple-minded, but can't you just create a zero matrix:

s = SparseArray[{}, {2, 5}, 0]


Then fill in the non-zero ones as required:

Table[s[[i, j]] = 2 i + j, {i, 1, 2}, {j, {1, 3, 5}}]
`