Functional-Style Partial Application and the Accumulator Pattern?

Following up on Functional-Style Fixed-Length Queue Object?, I want to generalize it to a functional-style accumulator pattern. Lots of things look like accumulators, loosely defined as binary functions that take a state-carrying object as a first parameter, a datum as a second parameter, and return the state-carrying object. Such an accumulator can be scanned, mapped, or (theoretically, pending answer to the cited question) folded over sequences or streams of data to yield interesting results. Some things that look like accumulators are statistics, filters, smoothers, estimators, hidden Markov models (Kalman, Viterbi), controllers, I am sure many more.

For elegance, I'm interested in a functional style. To keep the example small, consider the following:

newAccumulator[] := <|"runningSum" -> 0|>;
SetAttributes[accumulate, HoldFirst];
accumulate[accumulator_, datum_?NumberQ] :=
(accumulator[["runningSum"]] += datum;
accumulator);


This will just keep a running sum, but we could easily keep higher-order statistics and such a thing can be quite rich and useful.

We can use this easily to compute Gauss's Day-of-Reckoning Sum, for example;

$myAccumulator = newAccumulator[]; Scan[accumulate[$myAccumulator, #] &, Range[100]];
$myAccumulator  <|"runningSum" -> 5050|>  Now, it occurred to me, "wouldn't it be nice if we could write Scan[partial[accumulate,$myAccumulator], Range[100]]


where

SetAttributes[partial, HoldRest];
partial[f_, arg_] := (f[arg_,#]&);


which I can unit-test as follows

partial[f, x][y]

f[x, y]


and even generalize (just a little):

partial[f_, as__] := (f[Sequence @@ (Quiet @ Join[as, ##])] &);


and unit-test

partial[f, x, y][z, w]
partial[f, x][y]
partial[f, x][]

f[x, y, z, w]
f[x, y]
f[x]


But, lo and behold, it doesn't work on my example:

$myAccumulator = newAccumulator[]; Scan[partial[accumulate,$myAccumulator], Range[100]]
$myAccumulator  <|"runningSum" -> 0|>  This one has me clueless. EDIT: The HoldRest attribute on partial does not seem to make a difference. I put it in there out of habit -- side-effecting functions often work best when they hold their arguments against premature evaluation. 1 Answer The generalized definition of partial fails because the generated function looks like this: partial[accumulate,$myAccumulator]

(* accumulate[Sequence@@Quiet[Join[$myAccumulator,##1]]]& *)  Note how accumulate will be called with only a single argument. Since accumulate is HoldFirst, that argument will not be expanded into a sequence of arguments and therefore the sole two-argument definition of partial will never be applied. TracePrint reveals the full story: TracePrint[partial[accumulate,$myAccumulator][1]]

partial[accumulate,$myAccumulator][1] partial[accumulate,$myAccumulator]
partial
accumulate
accumulate[Sequence@@Quiet[Join[$myAccumulator,##1]]]& Function 1 (accumulate[Sequence@@Quiet[Join[$myAccumulator,##1]]]&)[1]
accumulate[Sequence@@Quiet[Join[$myAccumulator,1]]] accumulate accumulate[Sequence@@Quiet[Join[$myAccumulator,1]]]


The final accumulate expression is inert and $myAccumulator is never updated. The use of Sequence and Join to assemble the curried argument list is thus flawed. The original definition of partial was very close -- it can be extended like this: ClearAll@partial SetAttributes[partial, HoldRest] partial[f_, as__] := f[as, ##]&  This produces the correct results: partial[accumulate,$myAccumulator]

(* accumulate[$myAccumulator, ##1] & *)$myAccumulator = newAccumulator[];
Scan[partial[accumulate, $myAccumulator], Range[100]]$myAccumulator

(* <|runningSum->5050|> *)


Beware that this simple redefinition of partial still contains an evaluation leak. While the curried arguments are correctly preserved unevaluated (\$myAccumulator in this example), the additional arguments are evaluated before applying f (the datum argument in this case). For example:

partial[Hold, 1 + 1][2 + 2]

(* Hold[1 + 1, 4] *)


We can correct for this problem by a more elaborate definition:

ClearAll@partial
SetAttributes[partial, HoldAll]
partial[f_, as__] := Function[Null, f[as, ##], HoldAll]


Note how this last definition gives the returned function the HoldAll attribute so that none of the passed arguments are evaluated prematurely. So, now:

partial[Hold, 1 + 1][2 + 2]

(* Hold[1 + 1, 2 + 2] *)

• I think I see. The leak is not in the definition of partial itself, but in the definition of the function it returns. – Reb.Cabin Aug 18 '14 at 23:12
• @Reb.Cabin While my proposed revised definitions of partial were correct, my explanation as to why the original generalized definition failed was in error. Please see my corrected and expanded explanation. – WReach Aug 19 '14 at 4:07