The behaviour we see is due to a bug. Specifically, it is caused by an evaluation leak during the type-inference stage of query evaluation. (This analysis is current as of version 11.0.0.0).
Cause
The exhibited error message can be reproduced at will by evaluating the following expression which represents only a small portion of the full query execution:
TypeSystem`ResetTypeApplyCache[]
TypeSystem`TypeApply[
Hold[Evaluate[Symbol[#name]]] &
, { TypeSystem`Assoc[TypeSystem`Atom[String], TypeSystem`Atom[String], 1]}
]
(* >> OwnValues::sym: Argument Symbol[Str] at position 1 is expected to be a symbol.
TypeSystem`UnknownType
*)
A careful inspection of a trace of this evaluation reveals that the type inferencer is not taking enough care to prevent evaluation of the components of the Hold[...] expression as it recursively descends into it. This bug is somewhat understandable since the ability to "quote" code in Mathematica is idiomatic rather than a first class concept. This makes it fiendishly difficult to prevent all evaluation of arbitrary code. But fiendishly difficult does not mean impossible, so it should be possible (if tedious) for WRI to fix this problem.
Why Does The First Evaluation Fail?
By default, any dataset query will fail in the event that any message is issued during its evaluation. The TypeApply expression we examined issues an error message, so the query fails.
We can rewrite the original query to an equivalent form:
ds[All, Hold[Evaluate[Symbol[#name]]] &]
After doing so, we can add the FailureAction option to tell the query to proceed even in the event that a message is issued:
ds[All, Hold[Evaluate[Symbol[#name]]] &, FailureAction -> None]
If this expression is evaluated in a fresh kernel session we still get the error message but the expected result is returned.
Why Does The Second Evaluation Succeed?
The type inferencing process caches its results. As we have seen, the TypeApply expression discussed earlier still produces a result (TypeSystem`UnknownType) even though it issued an error message. This result is cached. On subsequent executions the cached result is used, so no further messages are issued and the query evaluates without incident.
We can cause the query to fail every time if we clear the type cache first:
TypeSystem`ResetTypeApplyCache[]
Hold[Evaluate[Symbol[#name]]] & /@ ds
(* ... error ... *)
Work-around
We can avoid the error by replacing Symbol with Symbol&[] in an effort to hide it from the type system:
TypeSystem`ResetTypeApplyCache[]
ds = Dataset@{<|"name" -> "Kuba"|>};
Hold[Evaluate[Symbol&[][#name]]] & /@ ds
(* Hold["Kuba"] *)
Obviously, this is a hack that is very specific to this case. Symbol@@{#name} could also be used. By this gimmick, we cause the inferencer to give up and return Unknown earlier in the process. Such hacks may become unnecessary (or even harmful) in future releases as the type inferencer evolves.
The trick of hiding the head of an expression is actually a reasonably common idiom to work around the lack of a bulletproof mechanism for quoting code. Common examples include:
Sequence@@{...} to prevent inappropriate sequence expansion due to the ambiguity as to whether Sequence[...] means to use a sequence within the executing code or within the computed result.With@@Hold[...] or Function@@{...} to prevent variable renaming during code generation
