Mathematica Stack Exchange is a question and answer site for users of Mathematica. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My task is quite simple; I have a table of data like

M = {{1,2}, {1,3}, {3,4}, {5,1}, {6,9}, {6,8}, {6,1}};

so the table is sorted with respect to the first entry of each tuple. What I want is basically to gather tuples with the same 'x' entry in a new one. M would look like this:

M = {{1,(2+3)/2}, {3,4}, {5,1}, {6,(9+8+1)/3}};

This can be possibly solved elegantly with some gathering and application of summation and counting, but since I'm a C speaker, I've taken the For loop as my first try:

Stringed = ConstantArray[{-1., -1.}, Dimensions[StoreFlat][[1]]]; (* this is
the array where all the data will be stored in. StoreFlat is the sorted but
"uncollected" input *)
Module[{i = 1, j = 1, k = 1, a = 1., f = 1.},
   For[i = 1, i <= Dimensions[StoreFlat][[1]], i++, (* loop over the number of tuples *)
    If[i == 1, (* first entry *)
     f = StoreFlat[[1, 1]];
     a = StoreFlat[[1, 2]];,
     If[f == StoreFlat[[i, 1]], (* a tuple exists already with this entry *)
       a += StoreFlat[[i, 2]];
       f = StoreFlat[[i, 1]]; (* this entry is larger than the former one *)
       a = StoreFlat[[i, 2]];
       j++; k = 1;
    Stringed[[j]] = {f, a/k}; (* output. the second entry is divided by the number of
entries. *)

I know that this is probably not the fastest way. But since my input is not too larger (190 000 tuples of a real, floating point value) this should do the job. Is anything wrong about my approach? On my laptop, this thing takes a lot of time (10 to 20 minutes, I guess - I cancelled it).

share|improve this question

marked as duplicate by Mr.Wizard May 16 '14 at 0:20

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Have a look at GatherBy. Your task can be implemented as a one-liner, and be fast. If you want to use the language effectively, you need to use its own idioms, rather than porting code from other languages you already know. – Leonid Shifrin Mar 2 '14 at 10:49
Thanks Leonid, I used Stringed = GatherBy[StoreFlat, #[[1]] &]; Map[{#[[1, 1]], Mean[#[[All, -1]]]} &, Stringed] to solve my problem! – Clemens Mar 2 '14 at 11:14
@Clemens Could you explain why should anyone answer your questions? You haven't upvoted any answer so far. If answers are not satisfactory you should explain why. Right? – Artes Mar 30 '14 at 22:02
I've upvoted Leonid Shifrin's answer right at the 2nd March! GatherBy was all I needed and I posted my solution. Hope that was not offending someone. – Clemens Mar 31 '14 at 6:14
up vote 6 down vote accepted

As L.S. said, using MMA like C is a path to poor performance.

Based on the OP content, I took the liberty of assuming you want the Mean of remaining tuple values appended to the first:

d = {{1, 2}, {1, 3}, {3, 4}, {5, 1}, {6, 9}, {6, 8}, {6, 1}};
Map[{#[[1, 1]], Mean[#[[All, 2]]]} &, GatherBy[d, First]]

test = RandomInteger[{1, 20}, {100000, 2}];
Map[{#[[1, 1]], Mean[#[[All, 2]]]} &, GatherBy[test, First]]; // Timing


N.B.: that time of 6/100th of a second for 100000 tuples is on a netbook...

A quick explanation of the workings: Everything (well, practically everything) in Mathematica revolves around lists. The GatherBy[...,First] takes the list of your tuples, and gathers them based on the value of the first elements, giving you a list of lists, each having contents of tuples with the same first element.

Then Map is used to go over that list, and for each entry (a list of tuples with same firsts), it takes the first element of the first entry (the leading result element), and then takes the second element of any and all members of the tuple list and computes the Mean. The two results make up a list which is progressively built up to the final result.

I'd recommend highly looking at the thread of common pitfalls here, and once you've gotten comfortable with the language, Leonid Shifrin's guide to advanced programming here.

share|improve this answer
very helpful answer +1 – ubpdqn Mar 2 '14 at 12:00
It's slightly quicker, at least on my PC with this data, to just use Mean /@ GatherBy[test, First] – Simon Woods Mar 2 '14 at 12:24
@SimonWoods: yes. I didn't write a super-terse form, figured list handling in baby-steps... +1 on comment/ – ciao Mar 2 '14 at 22:39
{ First @ First @ #, Mean @ Last @ Transpose @ #}& /@ GatherBy[ M, First]
{{1, 5/2}, {3, 4}, {5, 1}, {6, 6}}


{First @ #1, Mean @ #2}& @@@ Transpose /@ GatherBy[ M, First]

See (Apply (at the first level) - @@@, Map - /@, Slot - #)

share|improve this answer

An approach using Reap and Sow:

Last@Reap[Sow@@@(Reverse/@M),_, {#1,Mean@#2}&]

Thank you rasher for correcting my initial post (see comments)

share|improve this answer
Need Last@ on the front... and one of the few cases where Sow/Reap is slower. Pretty though! – ciao Mar 2 '14 at 11:38
@rasher yes...doing from iPhone so messy...sorry – ubpdqn Mar 2 '14 at 11:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.