Finding elements from a sparse matrix

Question

I am dealing sparse matrices of very large dimensions. I need to locate the positive and negative elements of the matrices and make lists of their corresponding positions. Of course I can write a program which scans every element and gives the output. But is it possible to do it in a neater way? for instance, to make a list of 1's, I used Position[S,1], but in future I need to get a list of positions of all entries which are >0 for example.

As suggested, I am giving an example of one of the smallest lists. For instance, I want to make a list of positions of elements which are >0.

S={{1,0,0,0,1,0,0,0,0,-1,0,0,0,-1,0,0,0,-1,0,0,0,0,-1,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},      
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {1,0,0,0,2,0,0,0,0,-1,0,0,0,-1,0,0,0,-1,0,0,0,0,-1,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {-1,0,0,0,-1,0,0,0,0,2,0,0,0,2,0,0,0,1,0,0,0,0,-1,0,0,0,0},       
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},       
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {-1,0,0,0,-1,0,0,0,0,2,0,0,0,2,0,0,0,1,0,0,0,0,-1,0,0,0,0},      
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},       
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {-1,0,0,0,-1,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,-1,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,1},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},         
   {-1,0,0,0,-1,0,0,0,0,-1,0,0,0,-1,0,0,0,-1,1,0,0,0,2,0,0,0,1},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
   {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,1}};

Answer 1

Take a look at Unitize, UnitStep, Clip, Sign, etc. and the NonzeroPositions property of sparse arrays. Using these in combination with appropriate arithmetic operations you can quickly get positions.

You could also take a look at FilterRules, and use this against the array rules to get positions for specified elements.

With your updated post, e.g., elements >0:

SparseArray[UnitStep[Sign[S] - 1]]["NonzeroPositions"]

(*
{{1, 1}, {1, 5}, {5, 1}, {5, 5}, {10, 10}, {10, 14}, {10, 18}, {14, 
  10}, {14, 14}, {14, 18}, {18, 10}, {18, 14}, {18, 18}, {19, 
  19}, {19, 23}, {19, 27}, {23, 19}, {23, 23}, {23, 27}, {27, 
  19}, {27, 23}, {27, 27}}
*)

Using FilterRules:

   FilterRules[Reverse /@ ArrayRules[S], ele_ /; ele > 0][[All, 2]]

  (*
  {{1, 1}, {1, 5}, {5, 1}, {5, 5}, {10, 10}, {10, 14}, {10, 18}, {14, 
  10}, {14, 14}, {14, 18}, {18, 10}, {18, 14}, {18, 18}, {19, 
  19}, {19, 23}, {19, 27}, {23, 19}, {23, 23}, {23, 27}, {27, 
  19}, {27, 23}, {27, 27}}
  *)

If you're going to be doing various such searches on a given array, get the reversed rules once and use that result in the FilterRules to save time.

Answer 2

This is reasonably fast and straightforward:

SparseArray[Positive@S, Automatic, False]["NonzeroPositions"]

Update: Hmm, this slight modification of rasher's method is even faster on regular arrays:

SparseArray[1 - UnitStep[-S]]["NonzeroPositions"]

---

Timing comparison. The array is about 90% zeros, 5% positive, 5% negative.

Table[
   foo = RandomChoice[{0.9, 0.05, 0.05} -> {0, -2, 2}, {10^k, 100}];
   {SparseArray[UnitStep[Sign[foo] - 1]]["NonzeroPositions"]        // timeAvg,
    SparseArray[Positive@foo, Automatic, False]["NonzeroPositions"] // timeAvg,
    SparseArray[1 - UnitStep[-foo]]["NonzeroPositions"]             // timeAvg},
   {k, 3, 6}
 ] ~Prepend~ {"Sign", "Positive", "-foo"} // Grid

If foo or S is already a SparseArray, then rasher's Sign method is a bit faster than Positive. Oddly, the "-foo" method is now slowest.

Table[
   foo = SparseArray[
     RandomChoice[{0.9, 0.05, 0.05} -> {0, -2, 2}, {10^k, 100}]];
   {SparseArray[UnitStep[Sign[foo] - 1]]["NonzeroPositions"]        // timeAvg,
    SparseArray[Positive@foo, Automatic, False]["NonzeroPositions"] // timeAvg,
    SparseArray[1 - UnitStep[-foo]]["NonzeroPositions"]             // timeAvg}},
   {k, 3, 6}
 ] ~Prepend~ {"Sign", "Positive", "-foo"} // Grid

The timing function goes back to stackoverflow and Timo and Mr.Wizard.

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := 
  Do[If[# > 0.3, Return[#/5^i]] & @@  AbsoluteTiming@Do[func, {5^i}], {i, 0, 15}];

Answer 3

Here is an example that uses properties of SparseArray. A small test array is used to illustrate. positive gives the positions of positive values in the sparse array and negative gives the positions of the negative values in the sparse array. These can be sa = extracts using Extract[sa,positive] etc.

SparseArray[
  Thread[RandomInteger[{1, 100}, {100, 2}] -> 
    RandomReal[{-3, 3}, 100]], {100, 100}];
position = sa["NonzeroPositions"];
values = sa["NonzeroValues"];
positive = Extract[position, pst = Position[values, _?Positive]];
negative = 
  Extract[position, Complement[List /@ Range[Length[values]], pst]];

The random 100 x 100 sparse array is created. The values and positions of non-zero values extracted then positive values collected and complement is negative values.

There are a number of ways to do this.