This is an issue similar to the one discussed here, as part of the type system operations discussed here. In the case at hand, there are two distinct problems. The first is that in some cases the type system is too restrictive in that it will not accept unrestricted negative index arguments to StringTake. The second is that some forms of association key references are unrecognized by the type inferencer with the result that type-checking is turned off. One of these behaviours blocks us, while the other helps us out.
For the following examples, we will be using some TypeSystem` utilities:
Needs["TypeSystem`"]
For discussion purposes, we consider some slightly simplified cases. First, the bread-and-butter case, which works:
Dataset[<|"a" -> "ABC"|>][StringTake[#a, {1, 2}] &]
(* "AB" *)
Next, the problematic case:
Dataset[<|"a" -> "ABC"|>][StringTake[#a, {1, -1}] &]
(* Failure[...] *)
This fails because of a restriction placed upon the forms that the type inferencer will accept for the StringTake operator. The valid forms are:
TypeSystem`Signatures[StringTake]
(*
StringTake[Atom[String], Atom[Integer]] :> StringT
StringTake[Vector[Atom[String], n_], Atom[Integer]] :> Vector[StringT, n]
StringTake[Atom[String], {m_Integer, n_Integer}] /; m <= n :> StringT
StringTake[Vector[Atom[String], n_], {m_Integer, n_Integer}] /; m <= n :> Vector[StringT, n]
*)
Note that in cases involving a two-index second argument, the first index must be less than or equal to the first. When we use {1, -1}, we run afoul of this restriction.
If we try indices {-2, -1}, we conform to the rule and all is well:
Dataset[<|"a" -> "ABC"|>][StringTake[#a, {-2, -1}] &]
(* "BC" *)
But this does not explain why changing #a to #[a] makes the problematic expression work again:
Dataset[<|"a" -> "ABC"|>][StringTake[#["a"], {1, -1}] &]
(* "ABC" *)
The reason involves a subtle difference in the type inferencing workflow. To understand, let's begin by looking at the type of our data:
DeduceType[<|"a"->"ABC"|> ]
(* Assoc[Atom[String], Atom[String], 1] *)
Now, let's see what the type system thinks about accessing this type of association using a slot-syntax key reference:
TypeApply[#a&, {Assoc[Atom[String],Atom[String],1]}]
(* Atom[String] *)
It gets the correct answer. Thus, the valid form rules that we inspected above will be applied, and we see the successes and failures discussed above.
Contrast this with what happens when we try to access a key using function syntax:
TypeApply[#["a"]&, {Assoc[Atom[String], Atom[String], 1]}]
(* AnyType *)
The type system does not (as of 10.2.0) know what to do with this syntax. Therefore, it falls back to the very general type AnyType. Once a function application has been identified to have AnyType, all bets are off and the system makes no further attempt to trace types. Essentially, further type-checking is turned off. Function evaluation proceeds without consulting any further validity rules -- in particular the problematic rules discussed above. As a consequence, evaluation succeeds.
The type inferencer does recognize part-syntax:
TypeApply[#[["a"]] &, {Assoc[Atom[String], Atom[String], 1]}]
(* Atom[String] *)
So an expression using it will be fully type-checked, and will ultimately fail when it hits the index position restrictions:
Dataset[<|"a" -> "ABC"|>][StringTake[#[["a"]], {1, -1}] &]
(* Failure[...] *)
In summary, the distinct behaviours are caused by two restrictions within the type inferencing process: one that works against us, and one that works for us. I expect that these restrictions will be removed in future releases.
