2
$\begingroup$

I'll preface this by saying that I'm not a Mathematica veteran - I'm a chemist who is trying to find a way to import time-dependent spectral data in a usable form. I have figured out how to import the data in the following form (can't change):

data = {{blank, z1, z2, z3, ...},{x1, y11, y12, y13, ...},{x2, y21, y22, y23, ...},...}

Using various combinations of Drop[], Take[], and Transpose[] I can get the data separated into three slightly more usable lists:

zdata = {z1, z2, z3, ...}

xdata = {x1, x2, x3, ...}

ydata = {{y11, y12, y13, ...},{y21, y22, y23, ...},{y31, y32, y33, ...},...}

The ydata list obviously contains an arbitrary number of sublists (equal to the length of zdata) themselves of arbitrary length (equal to the the length of xdata). What I need is a way to construct a single list of the form:

xyzdata = {{{x1, y11, z1},{x2, y21, z1},{x3, y31, z1},...},{{x1, y12, z2},{x2, y22, z2},{x3, y32, z2},...},{{x1, y13, z3},{x2, y23, z3},{x3, y33, z3},...}

Basically, I have one set of x data, one set of z data, and many sets of y data, and I need to put these in xyz coordinate form, in xy sublists. I have no idea how to accomplish this - I can use Part[] and Transpose[] to construct the list:

yydata = {{y11, y21, y31...},{y12, y22, y32,...},...}

But then I'm stuck. Here's what I think: I need to add x to these such that:

xydata = {{{x1, y11},{x2, y21},{x3,y31},...},{x1, y12},{x2, y22},{x3, y32},...},...}

And then add z such that I arrive at my desired list, xyzdata.

Hopefully some list gurus can help me figure out how to PrePend[] xlist to ydata at the right level to get ydata - and then how to Append[] zdata to xydata at the right level to get xyzdata.

I hope that there exists a general approach that will work for an arbitrary number of y sets, and an arbitrary number of xy values per set. I looked pretty hard for previous solutions to this sort of problem but didn't find anything at all.

Hope I explained that well enough!

$\endgroup$
  • $\begingroup$ There seems to be an inconsistency. In data, the xs seem to correspond to the second index of y, whereas in xyzdata, the xs seem to correspond to the first index of the ys. Which is correct? I suspect the former, based on "The ydata list obviously contains an arbitrary number of sublists (equal to the length of zdata) ". $\endgroup$ – march Jul 14 '16 at 17:52
  • $\begingroup$ Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Also, please remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! $\endgroup$ – Michael E2 Jul 14 '16 at 17:53
  • $\begingroup$ You can format inline code and code blocks by selecting the code and clicking the {} button above the edit window. The edit window help button ? is also useful for learning how to format your questions and answers. You may also find this meta Q&A helpful $\endgroup$ – Michael E2 Jul 14 '16 at 17:53
  • $\begingroup$ @march you are correct - I've edited the post and it should be correct now. The first index of y should correspond to it's x value, and the second index of y should correspond to its z value. I'm learning that what I think I need is a way to combine two lists at a certain sublevel, like so: input: {1,2,3,4} and {{{a,b},{c,d}},{{e,f},{g,h}}} output: {{{1,a,b},{2,c,d}},{{3,e,f},{4,g,h}}} I think MapThread[] might work with Append but I have no idea how $\endgroup$ – Nate Jul 14 '16 at 18:09
  • $\begingroup$ @Nate. Thanks! I have already fixed my post according to your edits. I will proceed to expand on my post. $\endgroup$ – march Jul 14 '16 at 18:10
4
$\begingroup$

This is a fairly simple example, which you should be able to generalize over your larger problem (provided I understand you correctly):

xvals = {x1, x2, x3, x4, x5, x6, x7, x8};
yvals = {{y11, y12, y13}, {y21, y22, y23}, {y31, y32, y33}, {y41, y42,
    y43}, {y51, y52, y53}, {y61, y62, y63}, {y71, y72, y73}, {y81, 
    y82, y83}};
zvals = {z1, z2, z3};

Table[{xvals[[x]], yvals[[x, z]], zvals[[z]]}, {z, 1, 
  Length[zvals]}, {x, 1, Length[xvals]}]

(*{{{x1,y11,z1},{x2,y21,z1},{x3,y31,z1},{x4,y41,z1},{x5,y51,z1},{x6,y61,z1},{x7,y71,z1},{x8,y81,z1}},{{x1,y12,z2},{x2,y22,z2},{x3,y32,z2},{x4,y42,z2},{x5,y52,z2},{x6,y62,z2},{x7,y72,z2},{x8,y82,z2}},{{x1,y13,z3},{x2,y23,z3},{x3,y33,z3},{x4,y43,z3},{x5,y53,z3},{x6,y63,z3},{x7,y73,z3},{x8,y83,z3}}}*)
$\endgroup$
  • $\begingroup$ I didn't think of using Table - thank you, this works. $\endgroup$ – Nate Jul 14 '16 at 18:25
  • $\begingroup$ Yup, and if you have no issues extracting the data on your own as you originally did this looks to be the simplest solution for the more difficult part of recombining the lists. $\endgroup$ – user6014 Jul 14 '16 at 19:27
5
$\begingroup$

Method 1: From Extracted Lists

Here, we first extract the lists. Using

data = {{blank, z1, z2}, {x1, y11, y12}, {x2, y21, y22}, {x3, y31, y31}};

we get the data from

xdata = First /@ Rest@data
ydata = Rest /@ Rest@data
zdata = Rest@First@data
(* {x1, x2, x3} *)
(* {{y11, y12}, {y21, y22}, {y31, y31}} *)
(* {z1, z2} *)

Then:

MapThread[List, {
   Transpose@Table[xdata, {Length@zdata}],
   ydata,
   Table[zdata, {Length@xdata}]
  }, 2]

or

Transpose@Transpose[{
   Transpose@Table[xdata, {Length@zdata}],
   ydata,
   Table[zdata, {Length@xdata}]
  }, {3, 2, 1}]

or

Transpose@Map[Flatten, Thread /@ Thread[{Inner[List, xdata, ydata, List], zdata}], {2}]

Method 2: All at Once

This shortcuts all of the extracting x, y, and z lists and constructs the end-list all at once. We could use the methods above, nesting the definitions of the lists into the functions then called, but this was the first thing I came up with, it's already been explained, and it shows the behavior of some nice functions. Use the version above, though.

I will take as sample data,

data = {{blank, z1, z2}, {x1, y11, y12}, {x2, y21, y22}, {x3, y31, y31}};

Getting rid of the blank:

data[[1]] = Rest@data[[1]];

Then, we do

MapThread[Append, {Thread[{#1, {##2}}] & @@@ Rest@data, Table[First@data, {Length@data - 1}]}, 2]
(* { {{x1, y11, z1}, {x1, y12, z2}},
     {{x2, y21, z1}, {x2, y22, z2}},
     {{x3, y31, z1}, {x3, y31, z2}} } *)

Explanation

We'll take this in pieces. First of all, Rest@data picks out all but the first element of the list (i.e. all but the list of zs):

Rest@data
(* {{x1, y11, y12}, {x2, y21, y22}, {x3, y31, y31}} *)

We then want to take and "push" the xis onto the yijs:

Thread[{#1, {##2}}] & @@@ Rest@data
(* { {{x1, y11}, {x1, y12}},
     {{x2, y21}, {x2, y22}},
     {{x3, y31}, {x3, y31}} } *)

The function Thread[{#1, {##2}}] & takes a sequence of any number of elements long, takes the first element of the sequence and puts it in the #1 spot and takes the rest of the elements of the sequence and puts them in the ##2 spot. Then, Thread does the following:

Thread[f[a, {x, y, z}]]
(* {f[a, x], f[a, y], f[a, z]} *)

i.e. it "pushes" f onto every element of the list. As an example,

Thread[{#1, {##2}}] & @@ {x1, y11, y21}
(* {{x1, y11}, {x1, y21}} *)

The && is the (short-hand) operator form of Apply, which basically strips away the {}'s, i.e. this is the same as doing

Thread[{#1, {##2}}] & [x1, y11, y21]
(* {{x1, y11}, {x1, y21}} *)

Finally, @@@ is short-hand for Apply[stuff, 2] which applies the function above to each element of the list.

Step 2 involves "pulling" the zs onto the list. This one is actually easier to understand. We now have the intermediate result

intermediate = { {{x1, y11}, {x1, y12}}, {{x2, y21}, {x2, y22}}, {{x3, y31}, {x3, y31}} };

For each of the sub-lists, we need to Append a z1 to the first element and Append a z2 to the second element (and so on if we had more zs). To do this, I make a list of copies of the z-list that is the same length as intermediate:

First@data
zCopies = Table[First@data, {Length@data - 1}]
(* {z1, z2} *)
(* {{z1, z2}, {z1, z2}, {z1, z2}} *)

MapThread will "match" elements of two arrays of the same dimension and apply a function to those matched elements. For example, from the documentation,

MapThread[f, {{{a, b}, {c, d}}, {{u, v}, {s, t}}}, 2]
(* {{f[a, u], f[b, v]}, {f[c, s], f[d, t]}} *)

We need to do exactly this thing, where we need to Prepend the element from the second array to the element of the first array (which is a list). Thus:

MapThread[Append, {intermediate, zCopies}, 2]
(* { {{x1, y11, z1}, {x1, y12, z2}},
     {{x2, y21, z1}, {x2, y22, z2}},
     {{x3, y31, z1}, {x3, y31, z2}} } *)
$\endgroup$
  • $\begingroup$ I can't seem to get the dimensions to match up using this method - it runs into trouble at the Map[Append[] step. $\endgroup$ – Nate Jul 14 '16 at 18:26
  • $\begingroup$ I think in order to fix that, I would have to see an actual example of your data. I assumed a particular form of your list based on your post, but there might be extra elements floating around. Did you do the first step data[[1]] = Rest@data[[1]]? That gets rid of the blank and makes the first element of your list just the list of z values. $\endgroup$ – march Jul 14 '16 at 18:42
  • $\begingroup$ @Nate. I have a second method which uses the extracted lists (Method 2). Please see the updated post. Again, it assumes a certain form of your original list, but since you managed to extract the three lists already anyway, you can skip the first three lines and go directly to the last two operations. $\endgroup$ – march Jul 14 '16 at 18:48
  • $\begingroup$ thanks for your help - this is a good explanation that is easy to follow. One thing about the MMA language is that the abundance of shortcuts e.g. # &/@ is great for veterans, but makes it hard for newbies to parse good MMA code. I think I'm going with the Table[] method below for now, since it's easy and shouldn't take too long, but I appreciate you showing me this method as well since it will improve my code in the future. $\endgroup$ – Nate Jul 15 '16 at 14:18
  • 1
    $\begingroup$ @Nate just click the grey check mark to the left of the answer that was most useful to you! (Sounds like the Table version, which is the simplest!) $\endgroup$ – march Jul 15 '16 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.