I have the following complex nested iteration situation.

  • I begin with a list, data, that comprises 16 TemporalData objects

  • Each TemporalData object contains 2 paths. Let's call them v and r. They have a common date index, so the date paths in each path are both the same length.

enter image description here

  • I partition each of the paths in the TemporalData objects in data into 20 day periods, offset by 1 / overlapping by 19.
partitionedSets = 
  Partition[#, 20, 1]& /@ data[[#]]["DatePaths"] & /@ Range @ Length @ data;
  • partitionedSets now contains 16 lists of 2 lists (representing v and r) that vary in length as follows. vLengths and rLengths will always return the same values
vLengths = Length @ partitionedSets[[#, 1]] & /@ Range @ 16

(*{216, 216, 215, 214, 213, 212, 212, 212, 211, 210, 209, 208, 207,
207, 207, 206}*)
  • The output of the first list of partitionedSets looks like this:
partitionedSets[[1, All, 1]]

enter image description here

  • Now, for each of the 16 lists of v and r series, I want to create a Date indexed set of "zipped together" ordered pairs of v and r. This is easy enough to do manually. For example, to get the first set of orderedPairs from the first partitionedSet of the first TemporalData object in data:
orderedPairs = 
    {partitionedSets[[1, 1, 1, All, 1]], 
     Thread[{partitionedSets[[1, 1, 1, All, 2]], partitionedSets[[1, 2, 1, All, 2]]}]}]

enter image description here

The problem I have is when I attempt to gather all combinations of 16x2xn lists into lists of ordered pairs. This is because the value of n depends on the Length of the combination of v or r and the partitionedSet I am currently iterating through. I know the solution is to use Outer -- and I can get it to work if I arbitrarily limit n in this case to 200.

dumb = 
      {partitionedSets[[#1, 1, #2, All, 1]],
       Thread[{partitionedSets[[#1, 1, #2, All, 2]],
       partitionedSets[[#1, 2, #2, All, 2]]}]}]&, Range @ 16, Range @ 200];

enter image description here

But what I am really trying to achieve is to make this same code work where the value for the second slot #2 in Outer is dynamic / recursive (really should be something like Range @ vlengths[[#]]& /@ Range @ Length@vLengths — but this fails.)

So, my question is: How can I functionally create 16*n lists of orderedPairs where n is dynamic / nested /recursive?


1 Answer 1


You may forgo the bulk of your manipulations by using TimeSeriesWindow.

With example 2 path time series data

data = TemporalData /@
   FinancialData["GOOGL", {"High", "Low"}, {"2 Jan 2019", "10 Dec 2019"}]
   , FinancialData["GE", {"High", "Low"}, {"4 Jan 2019", "17 Dec 2019"}]

Then using Table to produce the indexed offsets and the "Dates" and "PathLength" properties of TemporalData the offset partitions can be created by

partitioned =
  With[{ts = #, dates = #["Dates"], pathlength = #["PathLength"], windowsize = 20},
     TimeSeriesWindow[ts, dates[[#]]] & /@
      Table[{i, i + (windowsize - 1)}, {i, 1, pathlength - (windowsize - 1)}]
     ] & /@ data;

Varying number of partitions have been created according to each TemporalData object.

Length /@ partitioned
{218, 221}

All 20 steps in length

DeleteDuplicates@Flatten@Map[#["PathLength"] &, partitioned, {2}]

with 2 paths

DeleteDuplicates@Flatten@Map[#["PathCount"] &, partitioned, {2}]

Your final form can be selected with the below. I am throwing in your Standardize request but not that it is best to put all requests in the original post as people don't respond well to additions. However, it is your first post so I am going to give you a break.

partitionedSets =
      List /@ #["Dates"]
      , List /@ Transpose@MapAt[Standardize, #["ValueList"], 1]
     ] &
   , partitioned
   , {All, All}


partitionedSets[[1, 1]]

displays your final structure for the first element of the first list with Standardize transform on first path.

Hope this helps.

  • $\begingroup$ This is really helpful and concise! Thank you very much for taking time to answer my question. As a note, if i wanted to apply a transform (say, Standardize) to /just the first path/ in each series once the partitioning into groups of 20 had occurred, what would you recommend? TimeSeriesThread? As this answers the broad question I will mark this as answered. Thank you again for your time and assistance! $\endgroup$
    – R110
    Commented Feb 23, 2021 at 21:19
  • $\begingroup$ Sorry, I missed something in my original comment and then I couldn't edit in time. Two things: (1) if i wanted to apply a transform (say, Standardize) to /just the first path/ in each series once the partitioning into groups of 20 had occurred, what would you recommend? TimeSeriesThread? (2) to answer this fully the final step is to convert each timeseries of two paths into a list of ordered pairs where the two values in each path are threaded together and combined with the common date index. ie {{Date(i),{v(i),r(i)},{Date(j),{v(j),r(j)}}. That's the part I'm really stuck on! Thank you! $\endgroup$
    – R110
    Commented Feb 23, 2021 at 21:31
  • $\begingroup$ @R110 See update. $\endgroup$
    – Edmund
    Commented Feb 23, 2021 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.