Extract data from ragged arrays

Question

I have data in ragged arrays of the form

$$\left( \begin{array}{cc} \text{x1} & \{\text{a1}\} \\ \text{x2} & \{\text{a2}\} \\ \text{x3} & \{\text{a3},\text{b3},\text{c3}\} \\ \text{x4} & \{\text{a4},\text{b4},\text{c4}\} \\ \text{x5} & \{\text{a5},\text{b5},\text{c5},\text{d5},\text{e5}\} \\ \text{x6} & \{\text{a6}\} \\ \text{x7} & \{\text{a7},\text{b7}\} \\ \text{x8} & \{\text{a8},\text{b8},\text{c8}\} \\ \text{x9} & \{\text{a9}\} \\ \end{array} \right)$$

and I wish to sort it into arrays such as $$\text{data1}=\left( \begin{array}{cc} \text{x1} & \text{a1} \\ \text{x2} & \text{a2} \\ \text{x3} & \text{a3} \\ \text{x4} & \text{a4} \\ \text{x5} & \text{a5} \\ \text{x6} & \text{a6} \\ \text{x7} & \text{a7} \\ \text{x8} & \text{a8} \\ \text{x9} & \text{a9} \\ \end{array} \right);$$

and $$\text{data2}=\left( \begin{array}{cc} \text{x3} & \text{b3} \\ \text{x4} & \text{b4} \\ \text{x5} & \text{b5} \\ \text{x7} & \text{b7} \\ \text{x8} & \text{b8} \\ \end{array} \right);$$

and $$\text{data3}=\left( \begin{array}{cc} \text{x3} & \text{c3} \\ \text{x4} & \text{c4} \\ \text{x5} & \text{c5} \\ \text{x8} & \text{c8} \\ \end{array} \right);$$

finally $$\text{data4}=\left( \begin{array}{cc} \text{x5} & \text{e5} \\ \end{array} \right);$$

The actual data is numeric and has many more rows and an unknown number of ragged columns. I have been trying with Cases but with no success so far. Any ideas? Thanks

***** Edit *****

Thank you all for your efforts. There is much to learn from.

I have had a go at timing the individual offerings.

ClearAll[kglr, u1066, syed1, syed2, lericr, eldo]

nn = 5000 {3, 4, 5, 6, 7};
data = Flatten[
   Table[{#[[1]], Rest[#]} & /@ 
     RandomReal[{-1, 1}, {nn[[i]], i + 1}], {i, Length@nn}], 1];

tkglr = Timing[raggedThread = Flatten[Map[Thread]@#, {2}] &;
    raggedThread@data;][[1]];
(*    *)
tu1066 = 
  Timing[Through[(Cases[{x_, #} :> {x, y}] & /@ 
          NestList[Prepend[_], {y_, ___}, 3])[data]];;][[1]];
(*    *)
tsyed1 = Timing[Flatten[Thread /@ data, {{2}, {1}}];][[1]];
tsyed2 = Timing[tlist = Thread /@ data;
    Clear[f];
    f[k_List, n_Integer] := 
     Scan[If[Length@# > n - 1, Sow[Part[#, n]], Nothing] &, k] // 
        Reap // Last // First;
    f[tlist, #] & /@ Range[Max@(Length /@ tlist)];][[1]];
(*    *)
tlericr = 
  Timing[Flatten[
      Inner[Thread@*List, data[[All, 1]], data[[All, 2]], 
       List], {2}];][[1]];
(*    *)
teldo = Timing[data1 = <|Rule @@@ data|>;
    get[n_] := KeyValueMap[List]@DeleteMissing@Query[All, n]@data1;
    {get[1], get[2], get[3], get[4], get[5]};][[1]];
(*    *)

The results are

TableForm[{tkglr, tu1066, tsyed1, tsyed2, tlericr, teldo}, 
 TableHeadings -> {{kglr, u1066, syed1, syed2, lericr, eldo}, None}]

$ \begin{array}{c|c} \text{kglr} & 0.140625 \\ \text{u1066} & 0.15625 \\ \text{syed1} & 0.140625 \\ \text{syed2} & 1.01563 \\ \text{lericr} & 0.171875 \\ \text{eldo} & 2.45313 \\ \end{array} $

So the methods of kglr and the first method of syed are the fastest.

Well done everyone

Answer 1

raggedThread = Flatten[Map[Thread] @ #, {2}]&;


pairlists = raggedThread @ list

{{{x1, a1}, {x2, a2}, {x3, a3}, {x4, a4}, {x5, a5},   
   {x6, a6}, {x7, a7}, {x8, a8}, {x9, a9}},   
 {{x3, b3}, {x4, b4}, {x5, b5}, {x7, b7}, {x8, b8}},   
 {{x3, c3}, {x4, c4}, {x5, c5}, {x8, c8}},   
 {{x5, d5}},   
 {{x5, e5}}}

Grid[MapThread[Prepend] @ {pairlists, "D" <> ToString[#] & /@ Range[5]}, 
 Dividers -> {{False, 2 -> True}, False}]

This also works:

raggedThread2 = Apply[Join[##, 2] &] @* Map[Map[List]] @* Map[Thread];

Answer 2

With @eldo's data:

list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
     c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
     b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};

---

Using Flatten:

Since Flatten can transpose ragged arrays:

Flatten[Thread /@ list, {{2}, {1}}] // Grid

---

Using Sow/Reap:

tlist= Thread/@list;

Clear[f];
f[k_List, n_Integer] := 
 Scan[If[Length@# > n - 1, Sow[Part[#, n]], Nothing] &, k] // Reap // 
   Last // First

f[tlist, #] & /@ Range[Max@(Length /@ tlist)] // Grid

---

Result

Answer 3

I think something like this might work:

(* This is my test data, since you didn't provide yours in copy-pasteable form, and I was too lazy to reproduce by hand. I did include an empty list in the ragged data as a robustness test case. *)
xs = {x1, x2, x3, x4};
data = {{a1}, {a2, b2, c2}, {}, {a4, b4}};

Flatten[MapThread[Thread@*List, {xs, data}], {2}]

(*
{{{x1,a1},{x2,a2},{x4,a4}},{{x2,b2},{x4,b4}},{{x2,c2}}}
*)

Not sure whether you want this form or if you want the transform of each sub-structure. You can map Transform over this if you want.

Alt

One character shorter, and might be clearer semantically...or not?

Flatten[Inner[Thread@*List, xs, data, List], {2}]

Old

Partition[Flatten[MapThread[Thread@*List, {xs, data}], {2, 3}], 2]

(*
{{{x1,x2,x4},{a1,a2,a4}},{{x2,x4},{b2,b4}},{{x2},{c2}}}
*)

This feels a bit hack-ish, and it seems like there should be a more elegant way.

Each of your dataN can be extracted from this.

Answer 4

 list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
    c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
    b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};

list // MatrixForm

 data = <|Rule @@@ list|>

<|x1 -> {a1}, x2 -> {a2}, x3 -> {a3, b3, c3}, x4 -> {a4, b4, c4}, x5 -> {a5, b5, c5, d5, e5}, x6 -> {a6}, x7 -> {a7, b7}, x8 -> {a8, b8, c8}, x9 -> {a9}|>

get[n_] := KeyValueMap[List] @ DeleteMissing @ Query[All, n] @ data

Grid[{MatrixForm /@ Table[get[i], {i, 5}]}]

get[1]

{{x1, a1}, {x2, a2}, {x3, a3}, {x4, a4}, {x5, a5}, {x6, a6}, {x7, a7}, {x8, a8}, {x9, a9}}

get[2]

{{x3, b3}, {x4, b4}, {x5, b5}, {x7, b7}, {x8, b8}}

etc.

Answer 5

data1 = Cases[list, {x_,{y_,___}}:>{x,y}]

(* {{x1,a1},{x2,a2},{x3,a3},{x4,a4},{x5,a5},{x6,a6},{x7,a7},{x8,a8},{x9,a9}} *) 

data2 = Cases[list, {x_,{_,y_,___}}:>{x,y}]

(* {{x3,b3},{x4,b4},{x5,b5},{x7,b7},{x8,b8}} *)

data3 = Cases[list, {x_,{_,_,y_,___}}:>{x,y}]

(* {{x3,c3},{x4,c4},{x5,c5},{x8,c8}} *)

data4= Cases[list, {x_,{_,_,_,y_,___}}:>{x,y}]

(* {{x5,d5}}  *) 

Or

{data1x,data2x,data3x,data4x} = Through[(Cases[{x_,#}:>{x,y}]&/@NestList[
  Prepend[_], {y_,___},3])[list]]

(* {
    {{x1,a1},{x2,a2},{x3,a3},{x4,a4},{x5,a5},{x6,a6},{x7,a7},{x8,a8},{x9,a9}},
    {{x3,b3},{x4,b4},{x5,b5},{x7,b7},{x8,b8}},
    {{x3,c3},{x4,c4},{x5,c5},{x8,c8}},
    {{x5,d5}}
   } 
*)

{data1,data2,data3,data4}=={data1x,data2x,data3x,data4x}

(* True *) 

list

(As given by @eldo):

list = {{x1, {a1}}, {x2, {a2}}, {x3, {a3, b3, c3}}, {x4, {a4, b4, 
c4}}, {x5, {a5, b5, c5, d5, e5}}, {x6, {a6}}, {x7, {a7, 
b7}}, {x8, {a8, b8, c8}}, {x9, {a9}}};