# Plotting reordered clusters with different colors and joined lines

I would like to show how typical days of energy load (found through clustering of a years worth of data) are spread throughout a year in a single graph with a color for each 'typical day' (or cluster).

What I have so far

Please excuse the probably sloppy coding as I am still getting used to Mathematica's way of thinking (previous R user and some python knowledge)

Here is the data in pastebin

I divided the daily load curves (365 * 24 measurements) into clusters :

loadclust =  FindClusters[load, num]


each list within a cluster represents a daily load curve within a year and each cluster doesn't contain the same amount of days.

To these daily loads (lists) I then joined a list which corresponds to the hours in a year (i.e. 1 to 24 for first day, 25 to 48 for second, etc...). I did this by finding the position of each daily list in the original ordered list, since I wanted to keep the different clusters in order.

clustyearh = Flatten[Table[
Table[
Join[
], 1];


Then I made pairs of the hour in the year and the load at that hour, while still maintaining the order of the clusters (so there are fours lists containing the paired coordinates for each cluster).

pairs = Table[
Transpose[
Join[{Flatten[
Transpose[
, {i, 4}];


giving a list of four lists of pairs {{{1, 1368}, ...{24, 1428}, {121, 1321}, ...}{clust2}{..}{..}}

Plotting

This is where I am having problems. First, when I used Listplot for pairs, it doesn't display all the points (I notice that it's the blue and red clusters especially missing). Second, when I include Joined -> True it joins the consecutive days in each cluster, but not in the year. If cluster one has days 1, 2, 5, 59 for example it will join those days.

Question

Is there a way to color each cluster within a line graph for the whole year. As an addition, I would love to be able to apply Manipulate to scroll through the year within a week or 2 week window.

Using Simon Wood's solution :

Create a list of indexes for the clustered days.

loadclust = FindClusters[load -> Range[365], 4];


Then make pairs of data to give as coordinates to a ListPlot, while adding data for a 25th hour for each day (i.e. the first hour of the next day) so the plot will be joined at all points :

With[{load2 = Append[load, {{0}}]},
pairs = Map[
Sequence @@
Thread[{24 (# - 1) + Range[25],


Create a manipulate graph to view the graph inside of a window (I have added the colors I originally used for each cluster).

Manipulate[
ListLinePlot[pairs, Frame -> True, Axes -> False,
PlotRange -> {{n, n + win - 1}, All},
PlotStyle -> {Blue, Orange, Red, Green}] /.
Line[pts_] :> Line[Split[pts, #2[[1]] - #1[[1]] == 1 &]], {{n, 1,
"First Hour"}, 1, 24*365 - win, 1}, {{win, 168, "Window Length"},2, 336, 1}]


Big thanks to Simon! Cheers, E

-
Welcome to Mathematica.SE! Please upload you data somewhere say using http://pastebin.com. I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2)Read the FAQs! 3) When you see good Q&A, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation. Remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign –  chris Nov 7 '12 at 18:05
It would be nice if on top of what @chris already asked for (i.e. data) you also post a sample of your desired output –  belisarius Nov 7 '12 at 20:38
@chris, thanks for the welcome. I've been digging through SE for a while, and lately especially Mathematica SE. I usually try and find my own solutions through search, trial, and error. Also, I don't usually consider myself good enough to contribute well, thus the lack of prior posts. I thought this one might be important enough though and I'll make sure to start contributing more. I'm not sure if I'm at liberty to share the data, but I can make an example load year using an average of the clusters. I will get that on my post ASAP, as well as a sample of desired output. –  Emy Nov 7 '12 at 21:46

It is probably easier to keep track of the days in each cluster if you use FindClusters like this:

loadclust = FindClusters[load -> Range[365], 4];


This returns the clusters in terms of day numbers instead of the full load curve for that day, e.g.

loadclust // Shallow


{{1, 5, 10, 15, 23, 26, 30, 32, 36, 41, <<75>>}, {2, 3, 6, 9, 11, 12, 27, 28, 29, 34, <<103>>}, {4, 8, 13, 14, 16, 20, 21, 22, 24, 25, <<85>>}, {7, 17, 18, 19, 33, 37, 58, 60, 73, 79, <<62>>}}

So the first cluster contains days 1, 5, 10 etc...

We can now replace each day number with the explicit list of {hour, load} pairs:

pairs = Map[Sequence @@ Thread[{24 (# - 1) + Range[24], load[[#]]}] &, loadclust, {2}];


The scrolling plot can be achieved by changing the PlotRange in a Manipulate. To prevent non-consecutive days from joining up you can use a replacement rule to Split each line wherever adjacent points are not 1 hour apart.

Manipulate[
ListLinePlot[pairs, Frame -> True, Axes -> False,
PlotRange -> {{n, n + win - 1}, All}] /.
Line[pts_] :> Line[Split[pts, #2[[1]] - #1[[1]] == 1 &]],
{{n, 1, "First Hour"}, 1, 24*365 - win, 1},
{{win, 168, "Window Length"}, 2, 336, 1}]


Update

The simplest way to get the lines to join up is to extend each day's data to 25 hours - i.e. including the first hour of the following day.

With[{load2 = Append[load, {{0}}]},
pairs = Map[
Sequence @@
Thread[{24 (# - 1) + Range[25],


Conceptually neater is to plot the whole year as a single data set, with a ColorFunction that changes day by day to reflect the cluster containing that day.

To do this we will need a function which takes a day as its argument, and returns a colour based on cluster membership. This defines 365 DownValues for daycolour:

MapIndexed[(daycolour[#1] = ColorData[1][#2[[1]]]) &, loadclust, {2}];


The data for the ListPlot is just the complete list of {hour,load} pairs for the whole year:

data = Thread[{Range[24*365], Flatten[load]}];


Here is the scrolling plot again, using the cluster-based ColorFunction:

Manipulate[
ListLinePlot[data, Frame -> True, Axes -> False,
PlotRange -> {{n, n + win - 1}, All},
ColorFunction -> (daycolour[Ceiling[#/24]] &),
ColorFunctionScaling -> False],
{{n, 1, "First Hour"}, 1, 24*365 - win, 1},
{{win, 168, "Window Length"}, 2, 336, 1}]


-
This is quite simply awesome. That's basically what I wanted. Is there any way to join two consecutive colors though? Here, when it switches color, there is a break in the line. –  Emy Nov 7 '12 at 23:47
@Emy If you add PlotStyle->ColorData[10] as an argument to ListLinePlot the colour table will be a rainbow so the next colour is close to the previous one. –  chris Nov 8 '12 at 8:28
Actually, I just added PlotStyle -> {Blue,Orange,Red, Green} as I had already used this color scheme for previous graphs of the clusters, and I wanted the colors to be different enough to really see where the clusters were situated. –  Emy Nov 8 '12 at 10:58
Thanks, that's perfect. I have accepted this answer as it is exactly what I was looking for. I used the 'ugly' solution because it produces a nicer plot. The neater solution gives a choppy plot. Is there any reason for this? Thanks again Simon. –  Emy Nov 8 '12 at 13:48
@Emy, there's a problem with antialiasing in graphics with VertexColors`, see here for example. –  Simon Woods Nov 8 '12 at 15:40