Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

What I would like to do is generate a histogram of the consecutive time spent under a level. For example if the level I was interested in was 1.0 and I had this data;

{{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},0.04375}, 
{{2010, 1, 1, 6, 45, 0.},0.04375}, {{2010, 1, 1, 7, 0, 0.},5},
{{2010, 1, 1, 7, 15, 0.}, 0.5},{2010, 1, 1, 7, 30, 0.}, 5}}}

I would get a histogram containing two values, 45 (7:00am - 6:15am) and 15 (7:30am - 7:15am) minutes.

Any help would be appreciated.

share|improve this question
add comment

2 Answers 2

up vote 3 down vote accepted

Split time-series data based on a one-place test function, and collect total durations for spells of periods satisfying the test condition:

 ClearAll[durationsF];
 durationsF[timeseries : {{{__}, _} ..}, test_] :=
    With[{ts =  timeseries /. {{dt__}, val_} :> {AbsoluteTime[{dt}], test[val]}},
    (Last@# - First@#) & /@  Map[First, Split[ts, #[[2]] === True &], {2}]/60 /. 
      (0) ->  Sequence[]];

OP's example:

 opdata = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.},  0.04375},
   {{2010, 1, 1, 6, 45, 0.},  0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5}, 
   {{2010, 1, 1, 7, 15, 0.},  0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};
 durationsF[opdata , # <= 1 &]
 (* {45, 15} *)

Weather data example:

 psdnWthr = WeatherData["Pasadena",  "Temperature", {{2012, 1, 1}, {2012, 1, 15}}];
 DateListPlot[psdnWthr, GridLines -> {Automatic, {15}}, Joined -> True]

enter image description here

 durationsF[psdnWthr, # <= 15 &]
 (* {180, 1200, 2460, 840, 480, 720, 120, 1980, 3840, 1020} *)

Notes: If the input time series is not sorted, use ts=SortBy[ts,First] in the function definition. For units of measurement other than minutes, multiply by appropriate factors.

share|improve this answer
    
Great. Execution time of 0.65 seconds on a 100,000 row data set is impressive. –  Cam Nov 20 '12 at 2:15
add comment

I assume that your information is sorted time-wise.

Save it as a symbol:

data = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.}, 0.04375}, {{2010, 1, 1, 6, 45, 0.}, 0.04375}, {{2010, 1, 1, 7, 0, 0.}, 5}, {{2010, 1, 1, 7, 15, 0.}, 0.5}, {{2010, 1, 1, 7, 30, 0.}, 5}};

Find the time spent on each step:

data2 = Prepend[
     #1, 
     DateDifference[First[#1], First[#2], "Minute"] // First
     ] & @@@ Partition[data, 2, 1]
(* {{15, {2010, 1, 1, 6, 15, 0.}, 0.04375}, {15, {2010, 1, 1, 6, 30, 0.},0.04375}, {15, {2010, 1, 1, 6, 45, 0.}, 0.04375}, {15, {2010, 1, 1, 7, 0, 0.}, 5}, {15, {2010, 1, 1, 7, 15, 0.}, 0.5}} *)

Split according to your criteria (level), then collect your desired result, in the format {start time, end time, time spent on level}

level = 1;
data3 = Module[{start, duration},
   Cases[Split[data2, Last[#1] < level && Last[#2] < level &], 
         x_ /; Last[Last[x]] < level :> 
            {
            start = x[[1, 2]],
            DatePlus[start, {duration = Total[First /@ x], "Minute"}], 
            duration
            }
        ]
   ]
(* {{{2010, 1, 1, 6, 15, 0.}, {2010, 1, 1, 7, 0, 0}, 45}, {{2010, 1, 1, 7, 15, 0.}, {2010, 1, 1, 7, 30, 0}, 15}} *)

Then, to plot it:

date = DateString[#, {"Hour", ":", "Minute"}] &;
BarChart[Last /@ data3, 
    ChartLabels -> ("From " <> date[#1] <> " to " <> date[#2] & @@@ Most /@ data3)]

Bar Chart

share|improve this answer
    
You might consider using AbsoluteTime[] instead and divide by sixty to get the number of minutes... –  J. M. Nov 19 '12 at 9:31
    
@J.M. Why? Is it faster? –  VF1 Nov 19 '12 at 16:42
    
It's usually easier to manipulate absolute times than dates. You'll need to do your own tests tho. –  J. M. Nov 19 '12 at 22:10
    
The data set I have has about 100,000 rows so I'm not sure how practical this is. On my machine I get 95 seconds to do the data2= line. –  Cam Nov 20 '12 at 2:13
    
@Cam as per J.M.'s suggestion, using AbsoluteTime and dividing by 60 speeds the whole thing up. –  VF1 Nov 20 '12 at 2:23
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.