Why does Part return empty list for non-existing elements in ;; specification?

Question

When using the ;; specification Part will return empty list for first index above a list dimensions. For example:

a = {{1, 2, 3}};

a[[2 ;;]]
(* {} *)

but

a[[2]]
Part::partw: Part 2 of {{1,2,3}} does not exist. >>

Is this a necessary feature for functions like Rest? Can the ;; behaviour be relied on or will it be changed to produce an error at some point?

Answer 1

Summary

A span of the form i+1 ;; i represents an empty span. When i == Length[a], then the fact that i+1 is greater than the length of the expression is tolerated in order to support this notation. a[2;;] is equivalent to a[2;;-1] and thus a[2;;1]: an empty span.

Details

A span i ;; j is defined to return the parts of an expression whose indices extend from i to j inclusively. If Span enforced the rule that j must always be strictly greater than or equal to i, then the smallest possible span would contain one element.

It is desirable to be able to express an empty span. Without the ability to express an empty span, then code would always have to be written with two distinct paths: one for non-empty spans and one for empty spans (discriminated between, for example, by an If expression).

An empty span is expressed by a span of the form i+1 ;; i. So, for example:

a = {x};

a[[1 ;; 1]]
(* {x} *)

a[[2 ;; 1]]
(* {} *)

The index 2 is permitted to be out of range in order to allow the expression of the empty span. 3, on the other hand, is invalid:

a[[3 ;; 1]]
(* Part::take: Cannot take positions 3 through 1 in {1}. >> *)

Coming back to the original example...

a[[2 ;; ]]
(* {} *)

... is equivalent to ...

a[[2 ;; -1]]
(* {} *)

... which is in turn equivalent to ...

a[[2 ;; 1]]
(* {} *)

... which is an explicit empty span specification.

The Curious Case of a[[0;;1]]

Consider this:

a[[0;;1]]
(* {} *)

For some reason, Mathematica considers the span 0;;1 to be empty. At first glance, this seems crazy. After all, part 0 usually refers to the head of an expression. So what does it even mean to take the span extending "from the head through to the first part"?

To understand this, we must first acknowledge that part extraction using Span always returns the head:

{1, 2}[[1;;1]]
(* {1, 2} *)

{1, 2}[[2;;1]]
(* {} *)

zot[1, 2][[1;;1]]
(* zot[1] *)

zot[1, 2][[2;;1]]
(* zot[] *)

So 0 in a span cannot be referring to the head. To what does it refer? The clue to the mystery lies in observing the following pattern:

b = {1, 2, 3};

b[[1;;3]] == b[[1;;-1]] == b[[-3;;3]] == b[[-3;;-1]] == {1, 2, 3}
b[[2;;3]] == b[[2;;-1]] == b[[-2;;3]] == b[[-2;;-1]] == {2, 3}
b[[3;;3]] == b[[3;;-1]] == b[[-1;;3]] == b[[-1;;-1]] == {3}
b[[4;;3]] == b[[4;;-1]] == b[[ 0;;3]] == b[[ 0;;-1]] == {}

All of these expressions return True. We see the use of "end-relative indexing", i.e. where negative indices count backwards from the end. Take special note of the last two subexpressions: b[[0;;3]] and b[[0;;-1]]. It is clear from the pattern that the index 0 plays the role of "one after the element indexed by -1", that is, "one after the last element".

So, in a Span, and only in a Span, the index 0 refers to the hypothetical part one element beyond the end of the structure. Just as 4 is tolerated as an index beyond the end, so is 0. It is a special case that allows end-relative indexing to be fully symmetric with the normal start-relative indexing, and to support empty spans.

Applying this reasoning to a, we find:

a[[0 ;; 1]] == a[[0 ;; -1]] == a[[2 ;; -1]] == a[[2 ;; 1]] == {}
(* True *)

Again, it must be emphasized that this special treatment of index 0 only occurs for spans. a[[0]] refers to the head of the expression (in this case, List).

Answer 2

Not so much of an answer, rather an extended comment. Somewhere on this site I saw someone explaining the numbering of parts of an expression. For example, consider an expression of the form f[a, b, c, d, e]. Part 0 is the head f, and parts 1 thru 5 are a, b, c, d, e. Negative part specifications are also allowed, thus the author of that explanation presented this visual aid:

     0
5    f    1
 e       a

 d       b 
4    c    2
     3

Or with negative parts:

      0
-1    f    -5
  e       a

  d       b 
-2    c    -4
     -3

It seems, that the part specification allows to wrap around to -6 or to 6. Observe what happens here:

a = {{1, 2, 3}};
{# ;; -1, a[[# ;;]]} & /@ Range[-2, 2] // TableForm  

-2;;-1    {{1,2,3}}[[-2;;All]]
-1;;-1         1    2   3
0;;-1   
1;;-1          1    2   3
2;;-1   

(the same result is returned if you replace # ;; -1 with # ;; All or # ;;).

The other way round:

{;; #, a[[;; #]]} & /@ Range[-2, 2] // TableForm

1;;-2 
1;;-1   1   2   3
1;;0    
1;;1    1   2   3
1;;2    {{1,2,3}}[[1;;2]]

However a[[3;;]] still returns an error. If you redefine a = {{a1}, {a2}}, then a[[3;;]] will return an empty list again.