2
$\begingroup$

What I would like to do is generate a histogram of the consecutive time spent under a level. For example if the level I was interested in was 1.0 and I had this data;

{{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},0.04375}, 
{{2010, 1, 1, 6, 45, 0.},0.04375}, {{2010, 1, 1, 7, 0, 0.},5},
{{2010, 1, 1, 7, 15, 0.}, 0.5},{2010, 1, 1, 7, 30, 0.}, 5}}}

I would get a histogram containing two values, 45 (7:00am - 6:15am) and 15 (7:30am - 7:15am) minutes.

Any help would be appreciated.

$\endgroup$
0

2 Answers 2

4
$\begingroup$

Split time-series data based on a one-place test function, and collect total durations for spells of periods satisfying the test condition:

 ClearAll[durationsF];
 durationsF[timeseries : {{{__}, _} ..}, test_] :=
    With[{ts =  timeseries /. {{dt__}, val_} :> {AbsoluteTime[{dt}], test[val]}},
    (Last@# - First@#) & /@  Map[First, Split[ts, #[[2]] === True &], {2}]/60 /. 
      (0) ->  Sequence[]];

OP's example:

 opdata = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},  0.04375},
   {{2010, 1, 1, 6, 45, 0.},  0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5}, 
   {{2010, 1, 1, 7, 15, 0.},  0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};
 durationsF[opdata , # <= 1 &]
 (* {45, 15} *)

Weather data example:

 psdnWthr = WeatherData["Pasadena",  "Temperature", {{2012, 1, 1}, {2012, 1, 15}}];
 DateListPlot[psdnWthr, GridLines -> {Automatic, {15}}, Joined -> True]

enter image description here

 durationsF[psdnWthr, # <= 15 &]
 (* {180, 1200, 2460, 840, 480, 720, 120, 1980, 3840, 1020} *)

Notes: If the input time series is not sorted, use ts=SortBy[ts,First] in the function definition. For units of measurement other than minutes, multiply by appropriate factors.

$\endgroup$
1
  • $\begingroup$ Great. Execution time of 0.65 seconds on a 100,000 row data set is impressive. $\endgroup$
    – Cam
    Commented Nov 20, 2012 at 2:15
3
$\begingroup$

I assume that your information is sorted time-wise.

Save it as a symbol:

data = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.}, 0.04375}, {{2010, 1, 1, 6, 45, 0.}, 0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5}, {{2010, 1, 1, 7, 15, 0.}, 0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};

Find the time spent on each step:

data2 = Prepend[
     #1, 
     DateDifference[First[#1], First[#2], "Minute"] // First
     ] & @@@ Partition[data, 2, 1]
(* {{15, {2010, 1, 1, 6, 15, 0.}, 0.04375}, {15, {2010, 1, 1, 6, 30, 0.},0.04375}, {15, {2010, 1, 1, 6, 45, 0.}, 0.04375}, {15, {2010, 1, 1, 7, 0, 0.}, 5}, {15, {2010, 1, 1, 7, 15, 0.}, 0.5}} *)

Split according to your criteria (level), then collect your desired result, in the format {start time, end time, time spent on level}

level = 1;
data3 = Module[{start, duration},
   Cases[Split[data2, Last[#1] < level && Last[#2] < level &], 
         x_ /; Last[Last[x]] < level :> 
            {
            start = x[[1, 2]],
            DatePlus[start, {duration = Total[First /@ x], "Minute"}], 
            duration
            }
        ]
   ]
(* {{{2010, 1, 1, 6, 15, 0.}, {2010, 1, 1, 7, 0, 0}, 45}, {{2010, 1, 1, 7, 15, 0.}, {2010, 1, 1, 7, 30, 0}, 15}} *)

Then, to plot it:

date = DateString[#, {"Hour", ":", "Minute"}] &;
BarChart[Last /@ data3, 
    ChartLabels -> ("From " <> date[#1] <> " to " <> date[#2] & @@@ Most /@ data3)]

Bar Chart

$\endgroup$
7
  • $\begingroup$ You might consider using AbsoluteTime[] instead and divide by sixty to get the number of minutes... $\endgroup$ Commented Nov 19, 2012 at 9:31
  • $\begingroup$ @J.M. Why? Is it faster? $\endgroup$
    – VF1
    Commented Nov 19, 2012 at 16:42
  • $\begingroup$ It's usually easier to manipulate absolute times than dates. You'll need to do your own tests tho. $\endgroup$ Commented Nov 19, 2012 at 22:10
  • $\begingroup$ The data set I have has about 100,000 rows so I'm not sure how practical this is. On my machine I get 95 seconds to do the data2= line. $\endgroup$
    – Cam
    Commented Nov 20, 2012 at 2:13
  • $\begingroup$ @Cam as per J.M.'s suggestion, using AbsoluteTime and dividing by 60 speeds the whole thing up. $\endgroup$
    – VF1
    Commented Nov 20, 2012 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.