This question is very similar to the one posted here: Interrogating a running evaluation but my running computation consists of a backtracking procedure which sows valid solutions. It is hard to tell when the computation will be over so I would like to take a look at the results Sow'ed so far. Is there any way to do this?


1 Answer 1


AFAIK, there is no way to get what you want with the built-in Reap and Sow.

However, here is a drop-in Reap - Sow replacement based on Internal`Bag structure, which is also the one that the actual Reap and Sow are based on:

SetAttributes[withSideEffect, HoldRest];
withSideEffect[code_, sideEffectCode_] := (sideEffectCode; code);

$storage = <||>

SetAttributes[reap, HoldFirst]

reap[expr_] := reap[expr, _]
reap[expr_, patt_] := reap[expr, patt, #2&]

(call : reap[expr_, patt_, func_]) /; !TrueQ[$inReap] :=
	  Block[{$storage = <||>, $inReap = True},

reap[expr_, patt:Except[_List], func_:Function[#2]] := 
  MapAt[First, reap[expr, {patt}, func], 2]

reap[expr_, patt_List, func_] :=
  Module[{result = expr, reaped, tag, matchingTags},
    {reaped, matchingTags} =
          With[{matchingTagsData = KeySelect[$storage, MatchQ[#]]},
            Sow[Keys @ matchingTagsData, tag];
              Map[Internal`BagPart[#, All]&]
            ] @ matchingTagsData
      If[matchingTags =!= {},
        KeyDropFrom[$storage, First @ matchingTags];
      {result, reaped}

ClearAll[sow, $globalTag]
sow[expr_] := sow[expr, $globalTag]
sow[expr_, _] /; !TrueQ[$inReap] := expr
sow[expr_, tags_List] := First @ Map[sow[expr, #]&, tags]
sow[expr_, tag_] := withSideEffect[expr,
	  If[!KeyExistsQ[$storage, tag],
    $storage[tag] = Internal`Bag[{expr}],
    (* else *)
    Internal`StuffBag[$storage[tag], expr]

getCurrentData[part: _Integer | {__Integer} | _Span | All : All] := 
  getCurrentData[part, $globalTag]

getCurrentData[part: _Integer | {__Integer} | _Span | All : All, tag_] := 
    Lookup[$storage, tag, {}],
    bag: Except[{}] :> Internal`BagPart[bag, part]

It can serve as an explanation of how Reap-Sow work, but it can also be used to access the sowed data at any time, something that built-in Reap-Sow can't do:

  sow[1, {h[1], h[2], g[1]}];
  sow[2, g[2]];
  Print["The current data for tag ", h[1], " is ", getCurrentData[h[1]]];
  sow[3, {h[1], g[2], g[3]}]
  {_h, _g}
  , f


 During evaluation of In[283]:= The current data for tag h[1] is {1}

 {3, {{f[h[1], {1, 3}], f[h[2], {1}]}, {f[g[1], {1}], f[g[2], {2, 3}], f[g[3], {3}]}}}

In terms of performance, these will be slower, but should not be too much slower than the built-ins, since they are based on the same underlying data structure.

  • 1
    $\begingroup$ Nice approach (and upvote). Also can be done using ResourceFunction["ExpressionBag"]. The main advantage to this is that documentation is available. $\endgroup$ Commented Nov 26, 2019 at 16:18
  • $\begingroup$ @DanielLichtblau Thanks. I was not aware of that one. But I have looked through the code of that one, and there is considerable difference. That one is basically a wrapper on top of expression bag structure. My code is a drop-in replacement for Reap / Sow (so, I don't need almost any documentation :).). I don't really expose expression bag here at all, and could in principle use different means to accumulate results . So my code is really solving a different (but related) problem. It is also several times shorter :) $\endgroup$ Commented Nov 26, 2019 at 18:42
  • $\begingroup$ @DanielLichtblau Also, quite frankly, that particular implementation doesn't strike me as exceptional - I see lots of bloat for very little added substance (w.r.t. the raw Internal`Bag API). Also, some really important points seem to have been missed there, for example that the original Internal`Bag is automatically garbage-collectable and thus does not require manual resource management - which is a big deal in practice, and is not the case for the ExpressionBag wrapper. $\endgroup$ Commented Nov 26, 2019 at 19:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.