# Histogram of runs

What I would like to do is generate a histogram of the consecutive time spent under a level. For example if the level I was interested in was 1.0 and I had this data;

{{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},0.04375},
{{2010, 1, 1, 6, 45, 0.},0.04375}, {{2010, 1, 1, 7, 0, 0.},5},
{{2010, 1, 1, 7, 15, 0.}, 0.5},{2010, 1, 1, 7, 30, 0.}, 5}}}


I would get a histogram containing two values, 45 (7:00am - 6:15am) and 15 (7:30am - 7:15am) minutes.

Any help would be appreciated.

-

Split time-series data based on a one-place test function, and collect total durations for spells of periods satisfying the test condition:

 ClearAll[durationsF];
durationsF[timeseries : {{{__}, _} ..}, test_] :=
With[{ts =  timeseries /. {{dt__}, val_} :> {AbsoluteTime[{dt}], test[val]}},
(Last@# - First@#) & /@  Map[First, Split[ts, #[[2]] === True &], {2}]/60 /.
(0) ->  Sequence[]];


OP's example:

 opdata = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},  0.04375},
{{2010, 1, 1, 6, 45, 0.},  0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5},
{{2010, 1, 1, 7, 15, 0.},  0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};
durationsF[opdata , # <= 1 &]
(* {45, 15} *)


Weather data example:

 psdnWthr = WeatherData["Pasadena",  "Temperature", {{2012, 1, 1}, {2012, 1, 15}}];
DateListPlot[psdnWthr, GridLines -> {Automatic, {15}}, Joined -> True]


 durationsF[psdnWthr, # <= 15 &]
(* {180, 1200, 2460, 840, 480, 720, 120, 1980, 3840, 1020} *)


Notes: If the input time series is not sorted, use ts=SortBy[ts,First] in the function definition. For units of measurement other than minutes, multiply by appropriate factors.

-
Great. Execution time of 0.65 seconds on a 100,000 row data set is impressive. – Cam Nov 20 '12 at 2:15

I assume that your information is sorted time-wise.

Save it as a symbol:

data = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.}, 0.04375}, {{2010, 1, 1, 6, 45, 0.}, 0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5}, {{2010, 1, 1, 7, 15, 0.}, 0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};


Find the time spent on each step:

data2 = Prepend[
#1,
DateDifference[First[#1], First[#2], "Minute"] // First
] & @@@ Partition[data, 2, 1]
(* {{15, {2010, 1, 1, 6, 15, 0.}, 0.04375}, {15, {2010, 1, 1, 6, 30, 0.},0.04375}, {15, {2010, 1, 1, 6, 45, 0.}, 0.04375}, {15, {2010, 1, 1, 7, 0, 0.}, 5}, {15, {2010, 1, 1, 7, 15, 0.}, 0.5}} *)


Split according to your criteria (level), then collect your desired result, in the format {start time, end time, time spent on level}

level = 1;
data3 = Module[{start, duration},
Cases[Split[data2, Last[#1] < level && Last[#2] < level &],
x_ /; Last[Last[x]] < level :>
{
start = x[[1, 2]],
DatePlus[start, {duration = Total[First /@ x], "Minute"}],
duration
}
]
]
(* {{{2010, 1, 1, 6, 15, 0.}, {2010, 1, 1, 7, 0, 0}, 45}, {{2010, 1, 1, 7, 15, 0.}, {2010, 1, 1, 7, 30, 0}, 15}} *)


Then, to plot it:

date = DateString[#, {"Hour", ":", "Minute"}] &;
BarChart[Last /@ data3,
ChartLabels -> ("From " <> date[#1] <> " to " <> date[#2] & @@@ Most /@ data3)]


-
You might consider using AbsoluteTime[] instead and divide by sixty to get the number of minutes... – J. M. Nov 19 '12 at 9:31
@J.M. Why? Is it faster? – VF1 Nov 19 '12 at 16:42
It's usually easier to manipulate absolute times than dates. You'll need to do your own tests tho. – J. M. Nov 19 '12 at 22:10
The data set I have has about 100,000 rows so I'm not sure how practical this is. On my machine I get 95 seconds to do the data2= line. – Cam Nov 20 '12 at 2:13
@Cam as per J.M.'s suggestion, using AbsoluteTime and dividing by 60 speeds the whole thing up. – VF1 Nov 20 '12 at 2:23