Ok, I know the simple answer is to set some form of Hold attribute to the function but bear with me for a bit while I explain my motivation and why that is not quite what I want.

I have a collection of data that is naturally grouped together and some functions that operate on that data. To me, the obvious way to represent this is a struct-like data-type and an Association seemed perfect for the job. There are a couple of questions and answers describing this approach on this site.

Great, now I have an association as follows: (Just an example)

<|
  "Atom Names" -> {"N","C","O","C","H","H"},
  "Atom Nr"    -> {56,23,117,81,211,5},
  "Resname"    -> {ALA,ALA,TYR,LEU,GLY,GLY},
  "Bias Value" -> {1,5,1,5,1,1}
  "getRandomAtom" :> RandomChoice[{56,23,117,81,211,5}],
  "atomExists" :> (MemberQ[{"N","C","O","C","H","H"},#]&)
|>

I have some other functions that operate on this collection of data.

f1[a_Association, args__] := a["getRandomAtom"] + Total[a["Atom Nr"]]
f2[a_Association, atom_String] := If[a[atomExists][atom], a["Bias Value"] + 1]
(*etc*)

Ideally these function would not just accept any Association but only the 'correct' type of Association. I can ensure this in a couple of ways. Eg. create a helper function and use Condition or PatternTest to check if the Association has all the correct keys or much more simply, I can just wrap the entire Association with a inert head.

Protect[atomData]
atomData[<|
  "Atom Names" -> {"N","C","O","C","H","H"},
  "Atom Nr"    -> {56,23,117,81,211,5},
  "Resname"    -> {ALA,ALA,TYR,LEU,GLY,GLY},
  "Bias Value" -> {1,5,1,5,1,1}
  "getRandomAtom" :> RandomChoice[{56,23,117,81,211,5}],
  "atomExists" :> (MemberQ[{"N","C","O","C","H","H"},#]&)
|>]

And in my functions, I can just check the heads.

f3[a_atomData,args__] := "Do Stuff"

But I'd like to retain the functionality of Association transparently. We can somewhat achieve this through the use of upvalues.

atomData[a_Association][key_] := a[key];
atomData /: h_[atomData[a_Association],args___] := h[a,args] 
(*And some additional stuff*)
atomData /: ToString[atomData[a_Association]] := a["Atom Names"]
atomData[a_Association][] := a

As this question notes, this will not work for all functions (eg. Lookup) as they have the HoldAllComplete attribute. (Coincidentally, my motivation is almost exactly the same as the OP of that question)

Here comes the problem. Since I defined these upvalues, my functions which check for the Head atomData won't work anymore.

f3[myatomdata,3,4] (* myatomdata has head atomData *)

The upvalues associated with atomData will be applied first and this will result in f3[<|...(*underlying association*)...|>,3,4], which will not be evaluated as the first argument nolonger has the Head atomData.

SetAttributes[f3,HoldFirst]

won't help either, as in the function call f3[myatomdata,3,4], the evaluator will leave myatomdata alone, which means that it will have Head Symbol, and once again f3 will not be evaluated.

It seems that I have defeated my entire purpose by setting these upvalues. Is there a better way to do what I want?

I can think of 2 ways, both of which seem quite ugly.

1. Modify the upvalue definition to exclude certain functions. Something like

     atomData /: 
     (h : Except[f1|f2|f3])[layout[a_Association], args___] := h[a, args] 
   

This seems particularly inelegant, as I'd have to modify this every time I add another function that will use atomData.

2. Do the head checking yourself.

     SetAttributes[f3,HoldFirst]
     atomData /: (h : Except[Head])[layout[a_Association], args___] := h[a, args]
     f3[a_ /; Head[a]===atomData] := "Do Stuff"
   

The ideal solution would be a way to prevent just the upvalue from evaluating and leaving all others (OwnValues,DownValues etc.) alone.

PS. I'm also open to the idea that this whole approach is rubbish if someone can suggest a better way. I come from a background of C++, Java, and Python; Thinking of everything in terms of objects has been ingrained in me. Apologies for the long-winded explanation.