# How to implement JoinTo efficiently?

AppendTo is not efficient so I'm not quite sure of the best way to implement its analogue for joining (e.g. in python this would be extend function). So how can this be improved:

ClearAll[JoinTo];
SetAttributes[JoinTo,HoldFirst];
JoinTo[a_, b_List] := (a = Join[a,b];)

• need to account for cases where e.g. a is an integer or has some other head – Mike Honeychurch Jan 2 '16 at 23:36
• This will also make a copy of whatever it is replacing, hence will be inefficient if used in a loop. You might consider nesting List, as in a={a,b} or maybe use of Reap[...Sow[a]...], depending on what is the application at hand. – Daniel Lichtblau Jan 2 '16 at 23:56
• Shouldn't using an association be efficient (at least if beforehand the size of Join[a,b] were known so that values had just to be updated instead of inserted)? AssociateTo at least claims to offer efficient in place modification. – Sascha Jan 3 '16 at 0:01
• @Sascha Maybe...Depends on how many AssociateTo operations are done, and there may be an ordering issue (I think new things get added at the back and not the front). I'm not sure what algorithmic complexity to expect for adding new items; might be O(log n) for n=length of the association (which of course is much better than O(n)). Best I can suggest is that this approach be tried and timed. – Daniel Lichtblau Jan 3 '16 at 16:32
• @Daniel Lichtblau, I probably should have written my comment more clearly to convey that it contains two separate ideas: (1) updating values of associations instead of adding key-value pairs might be efficient (one could create an association with "empty" key-value pairs and update values as necessary; call it pre allocation if you will). (2) Just try out the build-in function AssociateTo to add key-value pairs and compare with list operations. – Sascha Jan 3 '16 at 16:42

Mapping AppendTo over the list of items to append escapes the copy inefficiency.

a = Range[5];
AppendTo[a, #] & /@ Range[6, 10];

a
(* {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} *)


Update with function

ClearAll[joinTo]
Attributes[joinTo] = {HoldFirst};
joinTo[s_Symbol, items_List] := Last@(AppendTo[s, #] & /@ items)

joinTo[a,Range[11,15]]
(* {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} *)


Hope this helps.

Update 2: Benchmarks

With joinTo in this post:

a = Range[10000];
joinTo[a, Range[10000]] // AbsoluteTiming // First
(* 0.837118 *)


With JoinTo (the OP's version):

ClearAll[JoinTo];
SetAttributes[JoinTo, HoldFirst];
JoinTo[a_, b_List] := (a = Join[a, b];)

b = Range[10000];
JoinTo[b, Range[10000]] // AbsoluteTiming // First
(* 0.000032 *)


To check if the codes are doing the same thing:

a == b
(* True *)

• I don't usually add benchmarks to people's answers unasked, but in this case I felt that it was important. When you use Join you only create one copy, regardless of the length of the list. When you use AppendTo the way that you do, you create n copies when appending a list of length n. – C. E. Jan 3 '16 at 7:02
• @Pickett Oh, I thought AppendTo was not making a copy but adding to the original list. Good to know. Thanks. – Edmund Jan 3 '16 at 13:00
• @Pickett You have shown it is much slower. How would you demonstrate that it is using more memory? – Edmund Jan 3 '16 at 13:03
• MaxMemoryUsed indicates that it is not using more memory, however the efficiency that M.R. and Daniel (in his comment) are talking about is not memory efficiency but how fast the computation runs in a loop. What we see here is exactly why it is common to hear that one should avoid AppendTo in favor of, for example, Sow/Reap, or linked lists. – C. E. Jan 3 '16 at 14:11
• @Pickett I see. Thanks. – Edmund Jan 3 '16 at 15:30

Allow to join more than two lists (double underscore behind b)

ClearAll[JoinTo, a, b];
SetAttributes[JoinTo, HoldFirst];
JoinTo[a_, b__List] := (a = Join[a, b];)

a = {1, 2};

JoinTo[a, Range@3, {99}]

a


{1, 2, 1, 2, 3, 99}

• this fails if e.g. a=1 or a=f[1,2,3] – Mike Honeychurch Jan 2 '16 at 23:55
• Yes, and it should fail - Try Join[1, {1, 2}] – eldo Jan 2 '16 at 23:59
• that was my point. join requires lists. the definition in the post does not confine a to a particular head therefore implies joining heads other than lists. AppendTo allows you to add an integer to a list, so mixed heads ...and so on – Mike Honeychurch Jan 3 '16 at 0:28
• This doesn't address the comment from Daniel on the copy inefficiency, any ideas? – M.R. Jan 3 '16 at 0:57
• @M.R. The copy inefficiency is inherent to the concept of JoinTo, not a particular implementation of it. – Szabolcs Feb 2 '16 at 8:18