Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I've been trying to separate large segments of DateList formatted data (years worth at a time) into individual days using the Split function.
The data has the form:

data = {{{y,m,d,h,m,s}, 154},{{y,m,d,h,m,s}, 157},...} 

and unfortunately there is not a uniform length of data per day (or I would have just partitioned it). This is what I've been trying (or variations of it):

splitData = Split[data,#2[[1,3]]>#1[[1,3]]&];

or

splitData = Split[data,Less[#1[[1,3]],#2[[1,3]]]];

and plenty of other versions, but it doesn't split where I need it to.

I thought if I compared the days in the DateList slot the data would split (as days of a month are increasing until the new month begins.

My current solution (which is very inefficient) pulls the dates out of data and applies Union, Union@data[[All,1]], then iterates through each data point comparing dates using nested tables. ... I know, a very poor method.

Anyone see how to us Split or another more efficient method to separate?

If you need test data to work with I've been using this instead of the real data (a much smaller set):

dpmo[month_, year_] := DateDifference[{year, month, 1}, {If[month == 12, year + 1, year], If[month == 12, 1, month + 1], 1}];
dates = Flatten[Table[Table[{2012, month, day, 0, 0, 0}, {day, 1, dpmo[month, 2012]}], {month, 1, 12}], 1];
data = Flatten[Table[Table[{dates[[m]], RandomInteger[{100, 250}]}, {100}], {m, 1, Length@dates}], 1];

The code generates the equivalent to a years worth of data, however there is a definitive set of data per day (100) which I do not have in the actual set. I didn't feel like adding that to the test data.

share|improve this question
add comment

2 Answers 2

up vote 7 down vote accepted

You can use Gather which will do all the work for you:

Gather[data, #1[[1, 1 ;; 3]] == #2[[1, 1 ;; 3]] &]

{{{{2012, 1, 1, 0, 0, 0}, 1}, {{2012, 1, 1, 0, 0, 1}, 2}, {{2012, 1, 1, 0, 1, 0}, 3}}, {{{2012, 1, 2, 0, 0, 0}, 4}, {{2012, 1, 2, 0, 0, 1}, 5}}, {{{2012, 1, 3, 0, 0, 0}, 6}}}

For an alternative way of generating a range of dates, should you need them, you could try this approach with a start date and a number of days required:

DatePlus[{2012, 12, 25, 0, 0, 0}, #] & /@ Range[0,9]

{{2012, 12, 25, 0, 0, 0}, {2012, 12, 26, 0, 0, 0}, {2012, 12, 27, 0, 0, 0}, {2012, 12, 28, 0, 0, 0}, {2012, 12, 29, 0, 0, 0}, {2012, 12, 30, 0, 0, 0}, {2012, 12, 31, 0, 0, 0}, {2013, 1, 1, 0, 0, 0}, {2013, 1, 2, 0, 0, 0}, {2013, 1, 3, 0, 0, 0}}

share|improve this answer
    
Definitely better than mine, if the data is not ordered and we want the data ordered. –  Mark McClure Sep 18 '12 at 15:58
    
I'm not sure which is best, if for some reason the dates aren't ordered then won't SplitBy fail to group values for the same day which aren't successive, just as Gather will only order the values according to their position in the data? Perhaps both need a pre(post)-sorting phase. Quick timing check seemed to suggest Gather had the edge on speed. –  image_doctor Sep 18 '12 at 16:32
1  
Yes, SplitBy would definitely not group things by date, if the dates were out of order. Depending on the application, we might or might not want this behavior. For example, we might want different groups colored differently. –  Mark McClure Sep 18 '12 at 16:37
    
I didn't check timing. My guess would be that SplitBy would be faster since it does no re-ordering. Could be wrong, though. –  Mark McClure Sep 18 '12 at 16:40
    
Thanks for the responses, Gather seems to be doing the job perfectly, and although it's a lot of data the system seems to be doing it with little trouble and since it only has to parse it once (instead of continually) I'm satisfied. –  MRN16 Sep 18 '12 at 17:09
show 1 more comment

I believe that SplitBy works. Using very simple data:

data = {
 {{2012,1,1,0,0,0},1},{{2012,1,1,0,0,1},2},{{2012,1,1,0,1,0},3},
 {{2012,1,2,0,0,0},4},{{2012,1,2,0,0,1},5},
 {{2012,1,3,0,0,0},6}
};
divided = SplitBy[data,#[[1,1;;3]]&];
Column[divided,Dividers -> All]

enter image description here

Note that SplitBy[data,f] splits between x1 and x2 when f[x1]=!=f[x2]. In the case here, I've taken #[[1,1;;3]]& which corresponds to the day portion of the date with data format.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.