I have a Dataset in the following form but with many more rows and a few other columns.

ds = 
  AssociationThread[{"Date", "R1","R2"}, 
      {DatePlus[DateObject@{2007, 12, 31}, {#, "EndOfMonth"}], 
       Sequence @@ RandomReal[{-0.5, 1}, 2]}
  ] & /@ Range[3]

enter image description here

I would like to apply Rest@FoldList[#1 (1 + #2/100) &, 100, #] & to columns "R1" and "R2" to get the following.

enter image description here

Currently I am destroying the dataset and then recreating it to get this result.

  {<|"Date" -> DatePlus[DateObject@{2007, 12, 31}, {#, "EndOfMonth"}]|> & /@ Range[3],
   Normal@ds[Rest@FoldList[#1 (1 + #2/100) &, 100, #] &, {"R1", "R2"}]}]

However, I would like not to have to do this because the columns that do not need to be folded need to be reincorporated into the result (e.g."Date" in the example). The ds[All, {"R1" -> f, "R2" -> f}] syntax operates on each column's row values individually. Is there some similar syntax for folding a column? Or a non destructive way of doing this?

Ideally I am looking to get the following but I can prepend the starting 100 value and date to the dataset if I can get the above.

enter image description here


Don't know if that's good enough:

fold = Rest@FoldList[#1 (1 + #2/100) &, 100, #] &;

ds[Transpose][{"R1" -> fold, "R2" -> fold}][Transpose]

enter image description here

| improve this answer | |
  • $\begingroup$ This is reasonable. The dataset isn't so huge that the double Transpose will be a problem. +1 while we wait for other solutions. $\endgroup$ – Edmund Jun 7 '16 at 13:32
  • $\begingroup$ @Edmund Do you possibly know why this is not working: ds[Transpose@*Query[{"R1" -> fold, "R2" -> fold}]@*Transpose]? $\endgroup$ – Kuba Jun 7 '16 at 13:35
  • $\begingroup$ It seems correct syntax. Odd that ds[Query[{"R1" -> fold, "R2" -> fold}]@*Transpose][Transpose] works and it doesn't. Does it have something to do with ascending and descending operators? $\endgroup$ – Edmund Jun 7 '16 at 13:39
  • $\begingroup$ @Edmund, as for your bonus: Prepend[ds, AssociationThread[{"Date", "R1", "R2"} -> {DateObject@{2007, 12, 31}, 100., 100.}]] works fine. $\endgroup$ – J. M.'s discontentment Jun 7 '16 at 13:49
  • 2
    $\begingroup$ @Kuba The expression based upon @* does not work due to a limitation in the type inferencer. The result of the first mapping query operator has type Unknown. The second Transpose does not accept an unknown data type. As usual, the operators work just fine if we take the type system out of the picture, e.g. ds // Normal // Query[...]. In the alternate formulation in your response, the type deducer (not inferencer) gets a chance to restore the missing type information enabling the second Transpose to function. Oh, and +1. $\endgroup$ – WReach Jun 7 '16 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.