15
$\begingroup$

Suppose I have two arrays of data, which I want to divide:

n = 1000;
a = RandomInteger[{1, 9}, {n, n}];
b = RandomInteger[{1, 9}, {n, n}];
First @ AbsoluteTiming @ (a/b)

0.637064

But suppose that two thirds of the values are actually zero. I'd want the ones dividing by zero to just remain zero.

mask = RandomChoice[{0, 0, 1}, {n, n}];
c = mask a;
d = mask b;
First @ AbsoluteTiming @ Quiet[c/d /. Indeterminate -> 0]

8.568857

Now it has become very slow. How can we speed this up?

Edit: Thanks for all the answers guys. Below are some timings:

timings

| improve this question | |
$\endgroup$
8
$\begingroup$

SparseArray can help, given the size and nature of the mask. It's slightly faster to convert c and d to sparse arrays than to convert a and b.

mask = SparseArray@RandomChoice[{0, 0, 1}, {n, n}];

First@AbsoluteTiming[
 c = mask a;
 d = mask b;
 Quiet[foo2 = Block[{Indeterminate = 0}, SparseArray[c] / SparseArray[d]]]
 ]
(*
  0.151565
*)

Compare:

First@AbsoluteTiming[
  c = mask a;
  d = mask b;
  foo1 = Quiet[c/d /. Indeterminate -> 0]
  ]
(*
  6.492534
*)

foo2 == foo1
(*
  True
*)
| improve this answer | |
$\endgroup$
  • $\begingroup$ @kguler I'm not sure why you deleted your Block trick, which I've used because it's faster than any other trick I've tried. But let me know if there's a problem. My use of SparseArray is a significant difference, but Replace does not work on it. Block and Map were the only tools I had, and `Map was way too slow. $\endgroup$ – Michael E2 Oct 28 '14 at 4:37
  • 1
    $\begingroup$ The trick will fail when c and d are using different masks. (Block won't help because the FullForm of the output of ComplexInfinity generated by 1/0 etc. is DirectedInfinity[].) Anyway, it's the fastest method for OP's specific example so far. $\endgroup$ – xzczd Oct 28 '14 at 5:38
  • 2
    $\begingroup$ @xzczd Actually it seems very odd to have different masks. In the OP's Q, it's clear that Indeterminate is to go to 0. 1/0 does not become Indeterminate and there is no indication that ComplexInfinity should become 0. (I decided it probably should not.) $\endgroup$ – Michael E2 Oct 28 '14 at 12:15
  • $\begingroup$ I like this answer the best. It's a bit wordy, but clear what it is doing. It's fast and flexible (if you want to do more than a simple divide). It's not quite so fast if you want to do a Normal afterwards - about the same speed as the case with no zeroes. $\endgroup$ – wxffles Oct 28 '14 at 21:34
  • $\begingroup$ Michael, just saw your comment (somehow i think @name mechanism is broken; i don't get pinged when mentioned in comments). The only reason i deleted my answer was I forgot to refresh the kernel in a long/crowded mma session and got confused with conflicting results:) (+1) $\endgroup$ – kglr Oct 29 '14 at 3:05
8
$\begingroup$

Simply turn all the 0/0s into 0/1s:

ans2 = c/(1 - Sign@d + d); // AbsoluteTiming

{0.7630000, Null}

The above answer only works for specific example in the question i.e. it only handles positive lists and identical mask. For more general cases one can use:

n = 1000;
(* a and b contain non-positive elements now *)
origin := RandomInteger[{-9, 9}, {n, n}]
a = origin; b = origin;

(* c and d now use different masks *)    
mask := RandomChoice[{0, 0, 1}, {n, n}];
c = mask a; d = mask b;

ans2 = With[{s = Unitize@d}, Divide[s c, 1 - s + d]]; // AbsoluteTiming

{0.606000, Null}

Notice that Divide is faster than /.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ The best general answer so far IMHO. $\endgroup$ – Mr.Wizard♦ Oct 28 '14 at 8:55
  • 1
    $\begingroup$ Note that you can use Unitize@d in place of Abs@Sign@d. $\endgroup$ – Mr.Wizard♦ Oct 28 '14 at 8:59
  • 1
    $\begingroup$ @Mr.Wizard It makes sense that one might know beforehand that 0/0 ought to be 0 - in any case, Indeterminate -> 0 is specified in the question. The question leaves open what should happen to a result like 1/0 or ComplexInfinity. One might want infinity, zero or a different result, such as a maximum value. The general answer here effectively takes the union of the masks, which is a plausible (even common) use-case. But I think one ought to be careful in stating which generalized problem it is a solution to. - +1 long before now. :) $\endgroup$ – Michael E2 Oct 28 '14 at 12:40
  • $\begingroup$ Clever and fast, but I'm going to wonder what the heck it is doing when I come back to read the code a few months later. Comments are for the weak. $\endgroup$ – wxffles Oct 28 '14 at 21:39
  • $\begingroup$ @wxffles I think the most probable obstacle for one to understand the code is unfamiliarity with Unitize and vectorization :) …OK, I've added a bit explanation, just a bit though. $\endgroup$ – xzczd Nov 1 '14 at 3:29
7
$\begingroup$ 
SetAttributes[f, Listable];
f@__ := 0
f[a_, b_] /; b != 0 := a/b
First@AbsoluteTiming[f[c, d];]

Edit

On the same spirit, this is 50% faster:

SetAttributes[g, Listable];
g[_, 0] := 0
g[a_, b_] := Divide[a, b]
First@AbsoluteTiming[g[c, d];]
| improve this answer | |
$\endgroup$
  • $\begingroup$ Unfortunately this is not particularly fast. $\endgroup$ – Mr.Wizard♦ Oct 28 '14 at 11:29
  • 1
    $\begingroup$ @Mr.Wizard Unfortunately, it looks a lot like your code too... what have you done to belisarius! :P $\endgroup$ – rm -rf♦ Oct 29 '14 at 5:10
  • $\begingroup$ @rm-rf I am taking over the world one syntax at a time. :p $\endgroup$ – Mr.Wizard♦ Oct 29 '14 at 5:36
2
$\begingroup$ 
c/(d /. (0 | 0. -> Infinity));
| improve this answer | |
$\endgroup$
2
$\begingroup$

For the given example where no values in b are zero before the mask one can use:

Divide[mask*a, b]

Note the use of explicit Divide for optimum performance. Timings:

(* your data *)

(r1 = Quiet[c/d /. Indeterminate -> 0]); // AbsoluteTiming // First
(r2 = Divide[mask*a, b]);                // AbsoluteTiming // First

r1 == r2

4.578262

0.212012

True

For the general problem I cannot beat xzczd's method.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ +! I keep forgetting about Divide.... $\endgroup$ – Michael E2 Oct 28 '14 at 12:44
  • $\begingroup$ How about Divide[maskb maska a, b]? It's only a bit faster. If all factors are converted to SparseArray, it's faster still, even including the conversion time. $\endgroup$ – Michael E2 Oct 28 '14 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.