3
$\begingroup$

Excuse me. This might be user specific question but I need your help. I have an indexed data, where positive integers are used for indexing data. I am trying to group data putting condition on index variable so that I can analyze group-wise data. But I don't know. Please guide me. Here is an example:

testdat = {{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {3, a31, 
   b31}, {4, a41, b41}, {2, a22, b22}, {3, a32, b32}, {3, a33, 
   b33}, {4, a42, b42}, {5, a51, b51}, {6, a61, b61}, {6, a62, 
   b62}, {5, a52, b52}, {5, a53, b53}, {5, b54, b54}};

If I do the following 

GatherBy[testdat, First]

it groups data using first column as index. However, I want to group data in which rows indexed 1 and 2 should be one group, rows indexed 3,4 and 5 should be another group and row indexed 6 should be another group. Here is how the result should look like:

{{{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {2, a22, b22}}, {{3, 
   a31, b31}, {3, a32, b32}, {3, a33, b33}, {4, a41, b41}, {4, a42, 
   b42}, {5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, 
   b54}}, {{6, a61, b61}, {6, a62, b62}}}

Another way to group the data would be to put rows indexed 1,2 and 3 into one group and rows indexed 4,5 and 6 into another group like in the following result:

{{{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {2, a22, b22}, {3, a31,
    b31}, {3, a32, b32}, {3, a33, b33}}, {{4, a41, b41}, {4, a42, 
   b42}, {5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, 
   b54}, {6, a61, b61}, {6, a62, b62}}};

This is just the theory. I need code to perform this job. I expect to hearing form Great guys form this community. Thank you in advance.

Improve this question
$\endgroup$

6 Answers 6

Reset to default
2
$\begingroup$

We can MapIndexed over a list of group lists to create an indexing function g, then compose this with the test function fn:

gatherInto1[dat_, fn_, groups__List] :=
  Module[{g},
    MapIndexed[(g[#] = #2[[1]]) &, {groups}, {2}];
    GatherBy[dat, g @ fn @ # &]
  ]

Example:

gatherInto1[testdat, First, {1, 2}, {3, 4, 5}, {6}]

{
 {{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {2, a22, b22}},
 {{3, a31, b31}, {4, a41, b41}, {3, a32, b32}, {3, a33, b33}, {4, a42, b42},
   {5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, b54}},
 {{6, a61, b61}, {6, a62, b62}}
}

In Mathematica 10 we can use GroupBy and Lookup. (Fred Simons referenced this but did not provide a complete implementation.)

gatherInto2[dat_, fn_, groups__List] :=
  With[{asc = GroupBy[dat, fn]},
    Join @@ Lookup[asc, #, {}] & /@ {groups}
  ]

Each function has its place.

  • gatherInto1 works with patterns whereas gatherInto2 does not.
  • gatherInto2 can be quite a bit faster than gatherInto1.

Timings:

big = RandomInteger[{1, 6}, {500000, 3}];

gatherInto1[big, First, {1, 2}, {3, 4, 5}, {6}] // AbsoluteTiming // First
gatherInto2[big, First, {1, 2}, {3, 4, 5}, {6}] // AbsoluteTiming // First

0.524

0.029
Improve this answer
$\endgroup$
1
  • $\begingroup$ As always, thank you so much, Mr. Wizard. $\endgroup$
    – ramesh
    Commented Feb 6, 2015 at 14:37
3
$\begingroup$ 
group = (Switch[#, 
                1 | 2 | 3, 1,
                _,         2
         ] &);


GatherBy[testdat, group[ #[[1]] ] & ]

{
   {{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {3, a31, b31}, {2, a22, b22},
    {3, a32, b32}, {3, a33, b33}}, 
   {{4, a41, b41}, {4, a42, b42}, {5, a51, b51}, {6, a61, b61}, {6, a62, b62}, 
    {5, a52, b52}, {5, a53, b53}, {5, b54, b54}}
}

and for oryginal request:

group = (Switch[#, 
                6,     1,
                1 | 2, 2, 
                _,     3] &);

{
    {{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {2, a22, b22}}, 
    {{3, a31, b31}, {4, a41, b41}, {3, a32, b32}, {3, a33, b33}, 
     {4, a42, b42}, {5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, b54}},
    {{6, a61, b61}, {6, a62, b62}}
 }
Improve this answer
$\endgroup$
4
  • $\begingroup$ @ Kuba, thank you for your time to answer this question. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 15:55
  • $\begingroup$ @rka no problem, is it useful? :) $\endgroup$
    – Kuba
    Commented Dec 11, 2014 at 16:15
  • $\begingroup$ @ I am trying to apply your code to my need but failing. Whether you could help me. I have data indexed form 40 to 1057. That is oldindex=Range[40,1057]. I have constructed newindex like newindex=Partition[oldindexindex, 3]; Now I want to group data according to new index. Any help $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 16:54
  • 1
    $\begingroup$ In general, if you can find a function f that is constant on each of your groups of indices, then you can use that function in GatherBy with argument #[[1]]. In this special case GatherBy[dataset, Floor[(#[[1]] - 1)/3] &] will work. $\endgroup$
    – Fred Simons
    Commented Dec 11, 2014 at 18:30
2
$\begingroup$

One of the ways is the following:

GatherBy[testdat, Position[{{1, 2}, {3, 4, 5}, 6}, #[[1]]][[1]] &]

(* {{{1, a11, b11}, {1, a12, b12}}, {{2, a21, b21}, {2, a22, b22}}, {{3, a31, b31}, {3, a32, b32}, {3, a33, b33}}, {{4, a41, b41}, {4, a42, b42}}, {{5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, b54}}, {{6, a61, b61}, {6, a62, b62}}}  *)

For your second gathering, you can replace the first argument of Position with {{1,2,3}, {4,5,6}}.

It might be that the following is not exactly what you are looking for, but it shows something of the flexability of working with Associations. Here I construct the association:

asc=Association[ (First[#]->Rest[#])& /@ testdat]

(* <|1->{a12,b12},2->{a22,b22},3->{a33,b33},4->{a42,b42},5->{b54,b54},6->{a62,b62}|> *)

And here are the values with keys 1,2,3:

Lookup[asc,Key /@ {1,2,3}]

(*  {{a12,b12},{a22,b22},{a33,b33}} *)
Improve this answer
$\endgroup$
3
  • $\begingroup$ @ Fred, thank you for your answer. What if I have a large data and index is too big to navigate. In fact, I have over 2m rows. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 15:22
  • $\begingroup$ @rka I have no experience at all with large data sets. But I understood that the function Lookup with Associations is very fast, and independent where you want to look in your data (beginning or end). I also understood that Associations are "memory hungry". My feeling is that if your data set fits in the memory of Mathematica as an Association, you will be better of with Lookup, of course depending of what you want to do. So I edited my answer showing how you could transform your testdat to an Association and how you could use LookUp. You might also look at the function GroupBy. $\endgroup$
    – Fred Simons
    Commented Dec 11, 2014 at 15:59
  • $\begingroup$ @ Fred, thank you fro your suggestion. I will work on your function and suggestion. Thanks again. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 16:01
2
$\begingroup$ 
Cases[testdat, {Alternatives[##], __}] & @@@ {{1, 2}, {3, 4, 5}, {6}}

Update:

based on your comment you can do it using two methods: prepare your index before

index=Range[1,6,2]

and then 

Cases[testdat, {Alternatives[##], __}] & @@@ {index}

you can also do it like this:

Flatten[Table[Cases[testdat, {i, __}], {i, 1, 6, 2}], 1]
Improve this answer
$\endgroup$
13
  • $\begingroup$ @ Algohi, thank you for your answer. What if I have a large data and index is too big to navigate. In fact, I have over 2M obs. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 15:21
  • $\begingroup$ if you have too big data then cases is your choose because it is fast. i don't get what you mean be index is too big to navigate? as long as you have the criteria to group the rows then you can use this method. $\endgroup$
    – Basheer Algohi
    Commented Dec 11, 2014 at 15:24
  • $\begingroup$ In your code, you have {{1, 2}, {3, 4, 5}, {6}} as criterion. Can we specify criteria as function than list? Like index is between two numbers: i<=index<=i+1, {i, 1, 6, 2}. Something like this. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 15:37
  • $\begingroup$ @ Algohi, thanks for your update. I think, this might serve my need. I am working on it thought on my real data set. $\endgroup$
    – ramesh
    Commented Dec 11, 2014 at 16:06
  • $\begingroup$ you mean newindex=Partition[oldindex, 3]? use index instead of {index} in my code. like this: Cases[testdat, {Alternatives[##], __}] & @@@newindex $\endgroup$
    – Basheer Algohi
    Commented Dec 11, 2014 at 16:38
2
$\begingroup$

Just for fun:

Join @@@ ({{1, 2}, {3, 4, 5}, {6}} /.Last@Reap[Sow[{##}, #1] & @@@ testdat, _, Rule])

yields:

{{{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {2, a22, b22}}, {{3, 
   a31, b31}, {3, a32, b32}, {3, a33, b33}, {4, a41, b41}, {4, a42, 
   b42}, {5, a51, b51}, {5, a52, b52}, {5, a53, b53}, {5, b54, 
   b54}}, {{6, a61, b61}, {6, a62, b62}}}
Improve this answer
$\endgroup$
1
  • $\begingroup$ thanks for taking time to answer my question. $\endgroup$
    – ramesh
    Commented Feb 6, 2015 at 14:36
0
$\begingroup$ 
list =
  {{1, a11, b11}, {1, a12, b12}, {2, a21, b21}, {3, a31, b31}, {4, 
    a41, b41}, {2, a22, b22}, {3, a32, b32}, {3, a33, b33}, {4, a42, 
    b42}, {5, a51, b51}, {6, a61, b61}, {6, a62, b62}, {5, a52, 
    b52}, {5, a53, b53}, {5, b54, b54}};

Using GroupBy and KeyTake

group[a_, b_] :=
 Catenate @ KeyTake[GroupBy[a, First], #] & /@ b


group[list, {{1, 2}, {3, 4, 5}, {6}}] // TableForm

enter image description here

group[list, {{1, 2, 3}, {4, 5, 6}}] // TableForm

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.