4
$\begingroup$

I have data in ragged arrays of the form

$$\left( \begin{array}{cc} \text{x1} & \{\text{a1}\} \\ \text{x2} & \{\text{a2}\} \\ \text{x3} & \{\text{a3},\text{b3},\text{c3}\} \\ \text{x4} & \{\text{a4},\text{b4},\text{c4}\} \\ \text{x5} & \{\text{a5},\text{b5},\text{c5},\text{d5},\text{e5}\} \\ \text{x6} & \{\text{a6}\} \\ \text{x7} & \{\text{a7},\text{b7}\} \\ \text{x8} & \{\text{a8},\text{b8},\text{c8}\} \\ \text{x9} & \{\text{a9}\} \\ \end{array} \right)$$

and I wish to sort it into arrays such as $$\text{data1}=\left( \begin{array}{cc} \text{x1} & \text{a1} \\ \text{x2} & \text{a2} \\ \text{x3} & \text{a3} \\ \text{x4} & \text{a4} \\ \text{x5} & \text{a5} \\ \text{x6} & \text{a6} \\ \text{x7} & \text{a7} \\ \text{x8} & \text{a8} \\ \text{x9} & \text{a9} \\ \end{array} \right);$$

and $$\text{data2}=\left( \begin{array}{cc} \text{x3} & \text{b3} \\ \text{x4} & \text{b4} \\ \text{x5} & \text{b5} \\ \text{x7} & \text{b7} \\ \text{x8} & \text{b8} \\ \end{array} \right);$$

and $$\text{data3}=\left( \begin{array}{cc} \text{x3} & \text{c3} \\ \text{x4} & \text{c4} \\ \text{x5} & \text{c5} \\ \text{x8} & \text{c8} \\ \end{array} \right);$$

finally $$\text{data4}=\left( \begin{array}{cc} \text{x5} & \text{e5} \\ \end{array} \right);$$

The actual data is numeric and has many more rows and an unknown number of ragged columns. I have been trying with Cases but with no success so far. Any ideas? Thanks

***** Edit *****

Thank you all for your efforts. There is much to learn from.

I have had a go at timing the individual offerings.

ClearAll[kglr, u1066, syed1, syed2, lericr, eldo]

nn = 5000 {3, 4, 5, 6, 7};
data = Flatten[
   Table[{#[[1]], Rest[#]} & /@ 
     RandomReal[{-1, 1}, {nn[[i]], i + 1}], {i, Length@nn}], 1];

tkglr = Timing[raggedThread = Flatten[Map[Thread]@#, {2}] &;
    raggedThread@data;][[1]];
(*    *)
tu1066 = 
  Timing[Through[(Cases[{x_, #} :> {x, y}] & /@ 
          NestList[Prepend[_], {y_, ___}, 3])[data]];;][[1]];
(*    *)
tsyed1 = Timing[Flatten[Thread /@ data, {{2}, {1}}];][[1]];
tsyed2 = Timing[tlist = Thread /@ data;
    Clear[f];
    f[k_List, n_Integer] := 
     Scan[If[Length@# > n - 1, Sow[Part[#, n]], Nothing] &, k] // 
        Reap // Last // First;
    f[tlist, #] & /@ Range[Max@(Length /@ tlist)];][[1]];
(*    *)
tlericr = 
  Timing[Flatten[
      Inner[Thread@*List, data[[All, 1]], data[[All, 2]], 
       List], {2}];][[1]];
(*    *)
teldo = Timing[data1 = <|Rule @@@ data|>;
    get[n_] := KeyValueMap[List]@DeleteMissing@Query[All, n]@data1;
    {get[1], get[2], get[3], get[4], get[5]};][[1]];
(*    *)

The results are

TableForm[{tkglr, tu1066, tsyed1, tsyed2, tlericr, teldo}, 
 TableHeadings -> {{kglr, u1066, syed1, syed2, lericr, eldo}, None}]

$ \begin{array}{c|c} \text{kglr} & 0.140625 \\ \text{u1066} & 0.15625 \\ \text{syed1} & 0.140625 \\ \text{syed2} & 1.01563 \\ \text{lericr} & 0.171875 \\ \text{eldo} & 2.45313 \\ \end{array} $

So the methods of kglr and the first method of syed are the fastest.

Well done everyone

$\endgroup$
2
  • 2
    $\begingroup$ Shouldn't there also be a {x5, d5}? $\endgroup$
    – lericr
    Sep 28, 2023 at 15:41
  • $\begingroup$ Yes, in my opinion $\endgroup$
    – eldo
    Sep 28, 2023 at 15:42

5 Answers 5

6
$\begingroup$
raggedThread = Flatten[Map[Thread] @ #, {2}]&;


pairlists = raggedThread @ list
{{{x1, a1}, {x2, a2}, {x3, a3}, {x4, a4}, {x5, a5},   
   {x6, a6}, {x7, a7}, {x8, a8}, {x9, a9}},   
 {{x3, b3}, {x4, b4}, {x5, b5}, {x7, b7}, {x8, b8}},   
 {{x3, c3}, {x4, c4}, {x5, c5}, {x8, c8}},   
 {{x5, d5}},   
 {{x5, e5}}}
Grid[MapThread[Prepend] @ {pairlists, "D" <> ToString[#] & /@ Range[5]}, 
 Dividers -> {{False, 2 -> True}, False}]

enter image description here

This also works:

raggedThread2 = Apply[Join[##, 2] &] @* Map[Map[List]] @* Map[Thread];
$\endgroup$
2
  • $\begingroup$ Ah, there it is. Map[Thread][...] instead of MapThread[...] (which is what I did). $\endgroup$
    – lericr
    Sep 28, 2023 at 21:44
  • $\begingroup$ @kglr Your method is the fastest. Thanks $\endgroup$
    – Hugh
    Oct 5, 2023 at 14:09
3
$\begingroup$

With @eldo's data:

list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
     c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
     b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};

Using Flatten:

Since Flatten can transpose ragged arrays:

Flatten[Thread /@ list, {{2}, {1}}] // Grid

Using Sow/Reap:

tlist= Thread/@list;

Clear[f];
f[k_List, n_Integer] := 
 Scan[If[Length@# > n - 1, Sow[Part[#, n]], Nothing] &, k] // Reap // 
   Last // First

f[tlist, #] & /@ Range[Max@(Length /@ tlist)] // Grid

Result

enter image description here

$\endgroup$
1
  • $\begingroup$ Your first method is the fastest. Equal to kglr. Thanks $\endgroup$
    – Hugh
    Oct 5, 2023 at 14:11
2
$\begingroup$

I think something like this might work:

(* This is my test data, since you didn't provide yours in copy-pasteable form, and I was too lazy to reproduce by hand. I did include an empty list in the ragged data as a robustness test case. *)
xs = {x1, x2, x3, x4};
data = {{a1}, {a2, b2, c2}, {}, {a4, b4}};

Flatten[MapThread[Thread@*List, {xs, data}], {2}]

(*
{{{x1,a1},{x2,a2},{x4,a4}},{{x2,b2},{x4,b4}},{{x2,c2}}}
*)

Not sure whether you want this form or if you want the transform of each sub-structure. You can map Transform over this if you want.

Alt

One character shorter, and might be clearer semantically...or not?

Flatten[Inner[Thread@*List, xs, data, List], {2}]

Old

Partition[Flatten[MapThread[Thread@*List, {xs, data}], {2, 3}], 2]

(*
{{{x1,x2,x4},{a1,a2,a4}},{{x2,x4},{b2,b4}},{{x2},{c2}}}
*)

This feels a bit hack-ish, and it seems like there should be a more elegant way.

Each of your dataN can be extracted from this.

$\endgroup$
1
  • $\begingroup$ Thanks. Not sure why this is slower than kglr's. Seems similar. $\endgroup$
    – Hugh
    Oct 5, 2023 at 14:14
2
$\begingroup$
 list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
    c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
    b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};

list // MatrixForm

enter image description here

 data = <|Rule @@@ list|>

<|x1 -> {a1}, x2 -> {a2}, x3 -> {a3, b3, c3}, x4 -> {a4, b4, c4}, x5 -> {a5, b5, c5, d5, e5}, x6 -> {a6}, x7 -> {a7, b7}, x8 -> {a8, b8, c8}, x9 -> {a9}|>

get[n_] := KeyValueMap[List] @ DeleteMissing @ Query[All, n] @ data

Grid[{MatrixForm /@ Table[get[i], {i, 5}]}]

enter image description here

get[1]

{{x1, a1}, {x2, a2}, {x3, a3}, {x4, a4}, {x5, a5}, {x6, a6}, {x7, a7}, {x8, a8}, {x9, a9}}

get[2]

{{x3, b3}, {x4, b4}, {x5, b5}, {x7, b7}, {x8, b8}}

etc.

$\endgroup$
2
$\begingroup$
data1 = Cases[list, {x_,{y_,___}}:>{x,y}]

(* {{x1,a1},{x2,a2},{x3,a3},{x4,a4},{x5,a5},{x6,a6},{x7,a7},{x8,a8},{x9,a9}} *) 
data2 = Cases[list, {x_,{_,y_,___}}:>{x,y}]

(* {{x3,b3},{x4,b4},{x5,b5},{x7,b7},{x8,b8}} *)
data3 = Cases[list, {x_,{_,_,y_,___}}:>{x,y}]

(* {{x3,c3},{x4,c4},{x5,c5},{x8,c8}} *)
data4= Cases[list, {x_,{_,_,_,y_,___}}:>{x,y}]

(* {{x5,d5}}  *) 

Or

{data1x,data2x,data3x,data4x} = Through[(Cases[{x_,#}:>{x,y}]&/@NestList[
  Prepend[_], {y_,___},3])[list]]

(* {
    {{x1,a1},{x2,a2},{x3,a3},{x4,a4},{x5,a5},{x6,a6},{x7,a7},{x8,a8},{x9,a9}},
    {{x3,b3},{x4,b4},{x5,b5},{x7,b7},{x8,b8}},
    {{x3,c3},{x4,c4},{x5,c5},{x8,c8}},
    {{x5,d5}}
   } 
*)
{data1,data2,data3,data4}=={data1x,data2x,data3x,data4x}

(* True *) 

list

(As given by @eldo):

list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};
$\endgroup$
1
  • 1
    $\begingroup$ Thanks. This was the approach I was thinking about too. Nice implementation. I couldn't figure out how to do the blanks programmaticly. $\endgroup$
    – Hugh
    Oct 5, 2023 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.