# Find continuous sequences inside a list

I have a list which is something like this:

{3,4,5,6,7,10,11,12,15,16,17,19,20,21,22,23,24,42,43,44,45,46}


What I'd like to to is get the intervals which are in a "continuous" sequence, something like:

{{3,7},{10,12},{15,17},{19,24},{42,46}}


and get the extremes. Note that the original data (of which this is a small excerpt) shows no sign of regularity or repetition. Numbers start from 1 and get up to 200 (these numbers come from applying Position[] to an array).

Any pointers/ideas?

-
–  Mr.Wizard Apr 19 at 13:56

You can use Split in this simple case

list = {3, 4, 5, 6, 7, 10, 11, 12, 15, 16, 17, 19, 20, 21, 22, 23, 24, 42, 43, 44, 45, 46};
{Min[#], Max[#]} & /@ Split[list, #2 - #1 == 1 &]


What it does is that the last argument to split gives True only when neighboring elements have a difference of 1. If not, the list is split there. Then you can use the Min/Max approach to find the ends. First and Last will work too.

Update:

Since the attention to this question/answer is rather surprising, let me point out one important thing: It is the crucial difference between Split and SplitBy. Both functions take a second argument to supply a testing function to specify the point to split but the behavior is completely different. Btw, the same is true for Gather and GatherBy.

While the second argument to Split makes that it

treats pairs of adjacent elements as identical whenever applying the function test to them yields True,

SplitBy does a completely different thing. It

splits list a into sublists consisting of runs of successive elements that give the same value when f is applied.

If you weren't aware of this, a closer look is surely advisable.

-
Awesome, thanks! I didn't know the Split[] function, now I know :) –  mag Apr 19 at 12:47
+1 Very clever! –  David Carraher Apr 19 at 14:02
@DavidCarraher Thanks. That many upvotes are rather surprising. –  halirutan Apr 20 at 9:43
I very nearly posted the same answer (except using [[All, {1,-1}]]) first, but the phone rang. Next time I'll let it ring! –  Simon Woods Apr 20 at 10:24
@SimonWoods It's a bit like digging for gold except the big rep pieces seem to be a mixture of easy but not directly solvable questions and topics which are of interest by a lot of people.. If you see such a question swimming by, let the phone ring ;-) –  halirutan Apr 20 at 10:36

Recently, I had to solve exactly the same problem. But my data consisted of several hundred lists of $10^6$ elements. The profiler showed this was becoming a significant overhead for my applications, so I invested an hour into a faster implementation. Anyway, here is a bit more than another order of magnitude improvement over Mr. W's answer (and more than 300× faster than the naive Mathematica implementation):

The algorithm is quite simple: Iterate through the list and every time a difference different from 1 appears (curr - prev) != 1 push prev as the closing part of the interval and curr as the opening part of the next interval.

InternalBag is used for O(1) insertion.

compiledGetContigIntervals =
Compile[{{ind, _Integer, 1}},
Block[{i, openInterval = 0, result = InternalBag[Most@{0}]},
openInterval = ind[[1]];(* the first opening interval *)
(* loop through all the indices and check for differences <> 1
If that is the case stuff the interval *)

Do[With[{curr = ind[[i]], prev = ind[[i - 1]]},
If[(curr - prev) != 1,
InternalStuffBag[result, openInterval];
InternalStuffBag[result, prev];
openInterval = curr;]]
, {i, 2, Length@ind}];

InternalStuffBag[result, openInterval];
InternalStuffBag[result, ind[[-1]]];
(* return the intervals *)
Partition[InternalBagPart[result, All], 2]],

"CompilationTarget" -> C, "RuntimeOptions" -> "Speed",
CompilationOptions -> {"ExpressionOptimization" -> True,
"InlineCompiledFunctions" -> True,
"InlineExternalDefinitions" -> True}];


(If you don't have a C compiler just leave out the options at the end of Compile)

And now the timings:

a = Delete[#, List /@ RandomSample[#, 500]] &@Range@1*^7;

intervals[a_List] := {a[[Prepend[# + 1, 1]]], a[[Append[#, -1]]]}\[Transpose] &@

(r1 = compiledGetContigIntervals[a]) // AbsoluteTiming // First
(r2 = intervals[a]) // AbsoluteTiming // First
(r3 = {Min[#], Max[#]} & /@ Split[a, #2 - #1 == 1 &]) // AbsoluteTiming // First
r1 === r3
r2 === r3

(*0.040002*)
(*0.191011*)
(*14.636837*)

(*True*)
(*True*)


(If the code is not compiled to C, but to the WVM the timing is 0.74 s)

-
Thanks to your answer I realized there was a much better approach to the problem than I had used. –  Mr.Wizard Apr 20 at 1:58
When you have time, could you add the Timing for my new method to your answer? I cannot test the fast C method in version 7 to compare. –  Mr.Wizard Apr 20 at 8:20
@Mr.Wizard Will do, but I'm afraid it will have to wait until Monday. –  Ajasja Apr 20 at 8:44
@Mr.Wizard Updated the timings –  Ajasja Apr 22 at 13:06
Thanks for the timings, and the edit. –  Mr.Wizard Apr 22 at 14:55

UPDATE: After reading Ajasja's answer I realized that I was making this way more complicated than it needed to be. My new code is easily an order of magnitude faster than my prior code or two orders faster than Split.

Split is a wonderfully clean method but it is not the fastest. Without resorting to C code one can get more than two orders of magnitude improvement on long lists with this:

intervals[a_List] :=
{a[[Prepend[# + 1, 1]]], a[[Append[#, -1]]]}\[Transpose] & @


Compared to Split:

a = Delete[#, List /@ RandomSample[#, 15000]] & @ Range@1*^7;

(r1 = intervals[a]) // Timing // First

(r2 = {Min[#], Max[#]} & /@ Split[a, #2 - #1 == 1 &]) // Timing // First

r1 === r2


0.0624

7.005

True

A short description of how the method works:

Differences is used to find the steps between each element and the next for the entire list.

I have used SparseArray Properties many times on this site: (1), (2), (3), (4), (5), (6), (7), (8), (9)
Here it is used as a well-optimized method to find the positions of all non-background elements in the differences list. I specify a background of 1 to find the positions of all other elements, representing a change of other than +1. (In later versions Pick is also well-optimized so that becomes an option(10) but here we need to manipulate the position list itself so it may be the best method even in later versions.)

Padding (Append, Prepend) the position list with 1 and -1 is used catch the first and last elements, respectively, of the original list. Adding (not appending) one to the list is used to offset the positions to get the elements on both sides of each jump. Finally, Transpose is used to pair off these values into the interval lists.

-
Spectacular, thanks! –  mag Apr 19 at 15:31
@mag I was able to make my code much faster; please take a look. And you're welcome. –  Mr.Wizard Apr 20 at 1:59
Could you include just a short description of how the code works? What does the "AdjacencyLists" do? Why the Append/Prepend? Don't have MMA at hand right now... –  Ajasja Apr 20 at 22:11
@Jacob I'll get v9 sometime soon, don't worry. Please follow the nine links provided; if after you still have questions I'll be happy to answer them to the best of my ability. –  Mr.Wizard Apr 22 at 0:05
@Mr.Wizard I didn't see it right away (its not in the docs), but I guess it somewhat speaks for itself :). I was expecting a SparseArray object, but I can handle lists :). –  Jacob Akkerboom Apr 22 at 0:13

A job for Mr Bray and Mr Curtis?

{First[#], Last[#]} & /@
FindClusters[{3, 4, 5, 6, 7, 10, 11, 12, 15, 16, 17, 19, 20, 21, 22,
23, 24, 42, 43, 44, 45, 46},
DistanceFunction -> BrayCurtisDistance]


{{3, 7}, {10, 12}, {15, 17}, {19, 24}, {42, 46}}

-
While interesting this appears to be highly inefficient; I hung my machine attempting to use this on longer lists. –  Mr.Wizard Apr 19 at 15:05

Not as neat as the other answers but yet another way :

alist = {3, 4, 5, 6, 7, 10, 11, 12, 15, 16, 17, 19, 20, 21, 22, 23, 24, 42, 43, 44, 45, 46};

{alist[[#[[1]]]], alist[[#[[2]] - 1]]} & /@ Partition[Union[{1, Length[alist] + 1},
1 + Position[Differences[alist], _?(# != 1 &)][[All, 1]]], 2, 1]

{{3, 7}, {10, 12}, {15, 17}, {19, 24}, {42, 46}}

-
I took the liberty of editing your answer. Revert if you dislike it. Due to the compressed formatting of the original I overlooked certain positive attributes. +1 –  Mr.Wizard Apr 20 at 10:42
@Mr.Wizard Many thanks, your edits are always useful. –  b.gatessucks Apr 20 at 11:12

Perhaps not as neat as some of the others, but a pedagogical use of Fold that I thought was worth sharing:

sequenceBoundaries[x : {__Integer}] :=
(Fold[ If[#2 - Last[Flatten[#1]] == 1,
{Sequence @@ Most[#1], Join[Last[#1], {#2}]},
Join[#1, {{#2}}]] &, {{First@x}},  Rest@x] )[[All, {1, -1}]]

sequenceBoundaries[{3, 4, 5, 6, 7, 10, 11, 12, 15, 16, 17, 19, 20, 21,
22, 23, 24, 42, 43, 44, 45, 46}]

(* {{3, 7}, {10, 12}, {15, 17}, {19, 24}, {42, 46}} *)
`
-