2
$\begingroup$

My task is quite simple; I have a table of data like

M = {{1,2}, {1,3}, {3,4}, {5,1}, {6,9}, {6,8}, {6,1}};

so the table is sorted with respect to the first entry of each tuple. What I want is basically to gather tuples with the same 'x' entry in a new one. M would look like this:

M = {{1,(2+3)/2}, {3,4}, {5,1}, {6,(9+8+1)/3}};

This can be possibly solved elegantly with some gathering and application of summation and counting, but since I'm a C speaker, I've taken the For loop as my first try:

Stringed = ConstantArray[{-1., -1.}, Dimensions[StoreFlat][[1]]]; (* this is
the array where all the data will be stored in. StoreFlat is the sorted but
"uncollected" input *)
Module[{i = 1, j = 1, k = 1, a = 1., f = 1.},
  Monitor[
   For[i = 1, i <= Dimensions[StoreFlat][[1]], i++, (* loop over the number of tuples *)
    If[i == 1, (* first entry *)
     f = StoreFlat[[1, 1]];
     a = StoreFlat[[1, 2]];,
     If[f == StoreFlat[[i, 1]], (* a tuple exists already with this entry *)
       a += StoreFlat[[i, 2]];
       k++;,
       f = StoreFlat[[i, 1]]; (* this entry is larger than the former one *)
       a = StoreFlat[[i, 2]];
       j++; k = 1;
       ];
     ];
    Stringed[[j]] = {f, a/k}; (* output. the second entry is divided by the number of
entries. *)
    ],
   ProgressIndicator@N[i/Dimensions[StoreFlat][[1]]]
   ]
  ];

I know that this is probably not the fastest way. But since my input is not too larger (190 000 tuples of a real, floating point value) this should do the job. Is anything wrong about my approach? On my laptop, this thing takes a lot of time (10 to 20 minutes, I guess - I cancelled it).

Improve this question
$\endgroup$
4
  • 5
    $\begingroup$ Have a look at GatherBy. Your task can be implemented as a one-liner, and be fast. If you want to use the language effectively, you need to use its own idioms, rather than porting code from other languages you already know. $\endgroup$
    – Leonid Shifrin
    Commented Mar 2, 2014 at 10:49
  • $\begingroup$ Thanks Leonid, I used Stringed = GatherBy[StoreFlat, #[[1]] &]; Map[{#[[1, 1]], Mean[#[[All, -1]]]} &, Stringed] to solve my problem! $\endgroup$
    – Clemens
    Commented Mar 2, 2014 at 11:14
  • $\begingroup$ @Clemens Could you explain why should anyone answer your questions? You haven't upvoted any answer so far. If answers are not satisfactory you should explain why. Right? $\endgroup$
    – Artes
    Commented Mar 30, 2014 at 22:02
  • $\begingroup$ I've upvoted Leonid Shifrin's answer right at the 2nd March! GatherBy was all I needed and I posted my solution. Hope that was not offending someone. $\endgroup$
    – Clemens
    Commented Mar 31, 2014 at 6:14

3 Answers 3

Reset to default
6
$\begingroup$

As L.S. said, using MMA like C is a path to poor performance.

Based on the OP content, I took the liberty of assuming you want the Mean of remaining tuple values appended to the first:

d = {{1, 2}, {1, 3}, {3, 4}, {5, 1}, {6, 9}, {6, 8}, {6, 1}};
Map[{#[[1, 1]], Mean[#[[All, 2]]]} &, GatherBy[d, First]]

test = RandomInteger[{1, 20}, {100000, 2}];
Map[{#[[1, 1]], Mean[#[[All, 2]]]} &, GatherBy[test, First]]; // Timing

(*
{{1,5/2},{3,4},{5,1},{6,6}}
{0.062400,Null}
*)

N.B.: that time of 6/100th of a second for 100000 tuples is on a netbook...

A quick explanation of the workings: Everything (well, practically everything) in Mathematica revolves around lists. The GatherBy[...,First] takes the list of your tuples, and gathers them based on the value of the first elements, giving you a list of lists, each having contents of tuples with the same first element.

Then Map is used to go over that list, and for each entry (a list of tuples with same firsts), it takes the first element of the first entry (the leading result element), and then takes the second element of any and all members of the tuple list and computes the Mean. The two results make up a list which is progressively built up to the final result.

I'd recommend highly looking at the thread of common pitfalls here, and once you've gotten comfortable with the language, Leonid Shifrin's guide to advanced programming here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ very helpful answer +1 $\endgroup$
    – ubpdqn
    Commented Mar 2, 2014 at 12:00
  • 4
    $\begingroup$ It's slightly quicker, at least on my PC with this data, to just use Mean /@ GatherBy[test, First] $\endgroup$
    – Simon Woods
    Commented Mar 2, 2014 at 12:24
  • $\begingroup$ @SimonWoods: yes. I didn't write a super-terse form, figured list handling in baby-steps... +1 on comment/ $\endgroup$
    – ciao
    Commented Mar 2, 2014 at 22:39
4
$\begingroup$ 
{ First @ First @ #, Mean @ Last @ Transpose @ #}& /@ GatherBy[ M, First]

{{1, 5/2}, {3, 4}, {5, 1}, {6, 6}}

or

{First @ #1, Mean @ #2}& @@@ Transpose /@ GatherBy[ M, First]

See (Apply (at the first level) - @@@, Map - /@, Slot - #)

Improve this answer
$\endgroup$
3
$\begingroup$

An approach using Reap and Sow:

Last@Reap[Sow@@@(Reverse/@M),_, {#1,Mean@#2}&]

Thank you rasher for correcting my initial post (see comments)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Need Last@ on the front... and one of the few cases where Sow/Reap is slower. Pretty though! $\endgroup$
    – ciao
    Commented Mar 2, 2014 at 11:38
  • $\begingroup$ @rasher yes...doing from iPhone so messy...sorry $\endgroup$
    – ubpdqn
    Commented Mar 2, 2014 at 11:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.