Select elements of list within a date range

Question

I have a list which consists of elements each of which has a date in DateList format, a number and a flag (Y/N). e.g.

{{{2010, 11, 29, 16, 6,  0.}, 17.7, "N"}, 
 {{2010, 12, 2,  14, 38, 0.}, 11.9, "N"}, 
 {{2010, 12, 2,  20, 1,  0.},  8.2, "N"}}

I want to extract those elements which are dated within the last 90 days and also the same date range but one year previous (into two separate lists).

I presume the way to do this would be to define an Interval between the AbsoluteTimes of the ends of the ranges. But how do I filter the list on this basis: i.e. apply a filter (IntervalMemberQ) on the basis of the AbsoluteTime of the first element in each element in the list? I presume that it's something to do with pure functions but I can't get the syntax right.

Answer 1

You have the logic right, so I'll show you an implementation based on it (I won't use IntervalMemberQ though, since it's simple enough without it).

To demonstrate, I'll slightly modify your input list to include dates from this year, since you said "within the last 90 days" (your dates are all from 2010, which means you must have a different reference point).

l = {{{2011, 12, 29, 16, 6, 0.}, 17.7, "N"}, 
     {{2012, 04, 29, 19, 6, 0.}, 17.7, "N"}, 
     {{2013, 5, 2, 14, 38, 0.}, 11.9,  "N"}, 
     {{2013, 2, 2, 20, 1, 0.}, 8.2, "Y"}};

selectElements[list_] := Module[{T = AbsoluteTime@DateList@Most@DateList[], T90d, T1y, T1y90d},
    {T90d, T1y, T1y90d} = T~DatePlus~# & /@ {-90, {-1, "Year"}, {{-1, "Year"}, {-90, "Day"}}};
    Select[list, Composition[T1y90d <= # <= T1y || T90d <= # <= T &, AbsoluteTime, First]] 
]

And now when you call this function on the input list, you get:

selectElements@l
(* {{{2011, 12, 29, 16, 6, 0.}, 17.7, "N"}, {{2013, 2, 2, 20, 1, 0.}, 8.2, "Y"}} *)

Based on the above, you can extend the definition of selectElements to select from within a custom date range as:

selectElements[list_, start_, end_] := Module[
    {s = AbsoluteTime@start, e = AbsoluteTime@end},
    Select[list, Composition[s <= # <= e &, AbsoluteTime, First]]
]

and you can use this as, say, selectElements[l, {2012,12,29}, {2013,2,23}].

Answer 2

ClearAll[prvsNdys1a];
prvsNdys1[data_, numdays_: 90, basedate_: DateList[], offset_: {0, "Year"}] := 
 With[{dtrange = DateRange[DatePlus[Take[basedate, 3], offset],
 DatePlus[Take[basedate, 3], {offset, {-numdays + 1, "Day"}}], {-1,  "Day"}]},
 Select[data, MemberQ[dtrange, First[#]] &]];

or

ClearAll[prvsNdys1b];
prvsNdys1b[data_, numdays_: 90, basedate_: DateList[],  offset_: {0, "Year"}] := 
 With[{at1 = AbsoluteTime[DatePlus[Take[basedate, 3], offset]], 
      at2 = AbsoluteTime[ DatePlus[Take[basedate, 3], {offset, {-numdays + 1, "Day"}}]]},
  Select[data, (at2 <= AbsoluteTime[#[[1]]] <= at1 &)]];

Example data:

dates = With[{first = Take[DateList[], 3]},
   DateRange[first, DatePlus[first, {-5 365, "Day"}], {-1, "Day"}]];
datalst = Transpose@{Sort@dates, Range[5 365 + 1], 
    RandomChoice[{"Y", "N"}, {5 365 + 1}]};
datalst[[;; 4]]
(*{{{2008, 3, 18}, 1, "Y"}, {{2008, 3, 19}, 2, "Y"}, {{2008, 3, 20}, 3, "Y"}, 
  {{2008, 3, 21}, 4, "N"}} *)

Usage examples:

Last 5 days:

prvsNdys1a[datalst, 5]
(* {{{2013, 3, 13}, 1822, "Y"}, {{2013, 3, 14}, 1823, "Y"},
   {{2013, 3, 15}, 1824, "Y"}, {{2013, 3, 16}, 1825,  "N"}, 
   {{2013, 3, 17}, 1826, "N"}}*)

Last 5 days and the corresponding days in previous 2 months:

prvsNdys1a[datalst, 5, DateList[], {-#, "Month"}] & /@ {0, 1, 2}
(* {{{{2013, 3, 13}, 1822, "Y"}, {{2013, 3, 14}, 1823, "Y"}, 
    {{2013, 3, 15}, 1824, "Y"}, {{2013, 3, 16}, 1825,  "N"},
    {{2013, 3, 17}, 1826, "N"}}, 
   {{{2013, 2, 13}, 1794, "Y"}, {{2013, 2, 14}, 1795, "N"}, 
    {{2013, 2, 15}, 1796, "N"}, {{2013, 2, 16}, 1797, "N"},
    {{2013, 2, 17}, 1798,  "N"}}, 
   {{{2013, 1, 13}, 1763, "N"}, {{2013, 1, 14}, 1764, "Y"},
    {{2013, 1, 15}, 1765, "N"}, {{2013, 1, 16}, 1766, "N"},
    {{2013, 1, 17}, 1767, "Y"}}}  *)

Last 3 days and corresponding days in the previous 3 years:

prvsNdys1a[datalst, 3, DateList[], {-#, "Year"}] & /@ Range[0, 4]
(*{{{{2013, 3, 15}, 1824, "Y"}, {{2013, 3, 16}, 1825, "N"}, {{2013, 3, 17}, 1826,"N"}}, 
   {{{2012, 3, 15}, 1459, "Y"}, {{2012, 3, 16}, 1460, "Y"},{{2012, 3, 17}, 1461, "Y"}}, 
   {{{2011, 3, 15}, 1093, "Y"}, {{2011, 3, 16}, 1094, "N"},{{2011, 3, 17}, 1095, "N"}}, 
   {{{2010, 3, 15}, 728,  "N"}, {{2010, 3, 16}, 729, "Y"},{{2010, 3, 17}, 730, "Y"}}, 
   {{{2009, 3, 15}, 363, "N"}, {{2009, 3, 16}, 364, "Y"}, {{2009, 3, 17}, 365, "Y"}}}*)

Alternative methods using Pick or Cases instead of Select:

Using Pick:

ClearAll[prvsNdys2a];
prvsNdys2a[data_, numdays_: 90, basedate_: DateList[], offset_: {0, "Year"}] := 
 With[{dtrange = DateRange[DatePlus[Take[basedate, 3], offset],
 DatePlus[Take[basedate, 3], {offset, {-numdays + 1, "Day"}}], {-1, "Day"}]},
 Pick[data, MemberQ[dtrange, #] & /@ First /@ data]];

or

 ClearAll[prvsNdys2b];
 prvsNdys2b[data_, numdays_: 90, basedate_: DateList[], offset_: {0, "Year"}] := 
  With[{at1 =UnitStep[ AbsoluteTime[DatePlus[Take[basedate, 3], offset]] - 
       AbsoluteTime /@ First /@ data], 
       at2 = UnitStep[ AbsoluteTime /@ First /@ data - 
       AbsoluteTime[DatePlus[Take[basedate, 3], {offset, {-numdays + 1, "Day"}}]] ]},
  Pick[data, at1 at2, 1]];

Using Cases:

ClearAll[prvsNdys3a];
prvsNdys3a[data_, numdays_: 90, basedate_: DateList[], offset_: {0, "Year"}] := 
 With[{dtrange = DateRange[DatePlus[Take[basedate, 3], offset],
 DatePlus[Take[basedate, 3], {offset, {-numdays + 1, "Day"}}], {-1, "Day"}]},
 Cases[data, {Alternatives @@ dtrange, _, _}]];