13
$\begingroup$

I am trying to use Sow/Reap to replace Append in my code. The problem is that I need to define the tags within my code, but doing so doesn't reap the desired results. A simplified version of my code is

Clear[tag];
Reap[tag = {a,b}; Sow[1, a];Sow[2,b];, tag]

However, if I define my tag outside of Reap, then it works

Clear[tag];
tag = {a,b};
Reap[Sow[1, a];Sow[2,b];, tag]

Any ideas as to how I can define tags within my code?

Improve this question
$\endgroup$
2
  • $\begingroup$ In short, the reason is the HoldFirst attribute of Reap. This attribute is necessary for Reap to be influence the evaluation of its first argument. $\endgroup$
    – Szabolcs
    Jun 4, 2012 at 15:59
  • $\begingroup$ @Szabolcs I think what is basically asked here is whether or not Reap can work with filtering tags which are only known when the code inside Reap is running. In such formulation, it makes perfect sense, but alas the answer is no. But one can always filter after the fact, and with a similar run-time performance (but possibly at the expense of memory use). $\endgroup$
    – Leonid Shifrin
    Jun 4, 2012 at 19:20

1 Answer 1

Reset to default
9
$\begingroup$

Reap does not work that way: when you provide a list of tags for Reap only these tags are "watched for" and collected. This allows for much better memory management than collecting for all tags and discarding at the end.

In essence it works something like this for declared tags 1, 2, 3:

{one, two, three} = {{},{},{}};

sow[x_, 1] := (one   = {one, x};   x)
sow[x_, 2] := (two   = {two, x};   x)
sow[x_, 3] := (three = {three, x}; x)
sow[x_, _] := x

SeedRandom[1]
Do[sow[i, RandomInteger[10^6]], {i, 10^7}]

Flatten /@ {one, two, three}

MaxMemoryUsed[]

{{30003, 1238414, 1529333, 3074569, 3401105, 4162839, 4715096, 5855206, 5971795, 6984287},
 {238730, 652982, 946353, 1821955, 2018277, 2065726, 4483637,
  4591412, 4733380, 4920935, 5283043, 5816356, 8272260, 8409277},
 {2793919, 2803799, 3784289, 5018439, 6380588, 8799862, 9301537}}

15034792

You can see that advance knowledge of the tags is needed to set this up. Compare to the memory requirements of this:

SeedRandom[1]

Cases[
 {#[[All, 1]], #[[1, 2]]} & /@
   Table[{i, RandomInteger[10^6]}, {i, 10^7}] ~GatherBy~ Last,
 {_, 1 | 2 | 3},
 {1}
]

MaxMemoryUsed[]

{{{30003, 1238414, 1529333, 3074569, 3401105, 4162839, 4715096, 
   5855206, 5971795, 6984287}, 1},
 {{238730, 652982, 946353, 1821955, 2018277, 2065726, 4483637, 
   4591412, 4733380, 4920935, 5283043, 5816356, 8272260, 8409277}, 2},
 {{2793919, 2803799, 3784289, 5018439, 6380588, 8799862, 9301537}, 3}}

605960144

If you wish to replace Append with something more efficient, consider using linked lists as I did in the sow example above, or the Internal`Bag class of functions.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.