A few years ago, I started getting deeper into Mathematica while doing mathematical physics. I stumbled upon this question on StackExchange. I found the answers very useful. I feel that after all these years I can add something to the existing responses based on my personal perspective and reflection. IMHO, at least one great was book mentioned only in passing: Roman E. Maeder's Computer Science with MATHEMATICA. If you are one of those people who fly on planes who don't want to know anything about the Navier-Stokes equation, please stop reading the rest of my post before you get the urge to downvote my answer.
The first milestone in understanding how MMA works under the hood for me was understanding John McCarthy's paper Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I. I found familiarity with Structure and Interpretation of Computer Programs by Harold Abelson, Gerald Jay Sussman, and Julie Sussman very useful. My next step was not a book but rather a series of three video lectures on Wolfram Programming Language Fundamentals by Professor Richard Gaylord.
- https://www.youtube.com/watch?v=H-rnezxOCA8
- https://www.youtube.com/watch?v=H-rnezxOCA8
- https://www.youtube.com/watch?v=u1Oijydu4qI
They are accompanied by a transcript. The indispensable parts of the transcript are pages 29-35 (Exercise). I can't recommend them enough.
If one is to read a single book on MAA internals, then the aforementioned David Wagner's Power Programming with Mathematica: The Kernel is perhaps "the best". However, even Dr. Wagner, in arguably capstone chapter 7 (Expression Evaluation) refers multiple times to chapter 10 (Rules-Based programming) of Roman E. Maeder's Computer Science with MATHEMATICA. Chapter 11 (Functions) and Chapter 12 (Theory of Computations) of Maeder's book take us back to John McCarthy's paper and theoretical research preceded it.
Dr. Maeder was doing a three-year post-doc in the department of computer science at the University of Illionis Urbana Champain when he became employee number two or three (depending if you count Dr. Wolfram as an employee or just the owner) of Wolfram Research. Dr. Maeder wrote a few other highly acclaimed books on Mathematica. Dr. Wagner was running his own consulting business at the time of writing the Power Programming book after resigning his associated professorship in theoretical computer science at the University of Colorado Boulder. Dr. Maeder is one of the principal architects of Mathematica. Dr. Wagner is John Lions of Mathematica (pun intended). Both of them had a primary interest in theoretical computer science and programming for the sake of programming rather than in doing scientific work (math-physics in my case) with Mathematica.
One of the earlier responses mentioned chapters 1,2,4, and 5 of Mathematica for Scientists and Engineers by Thomas B. Bahder. I have a copy on my desk. It most definitely those four chapters contains sufficient amount of MMA core programming for 99% of users with scientific/engineering background. However, it also contains still very readable chapters on Symbolic Calculations (Chapter 6), Numerical Calculations(Chapter 7), Vectors, Matrices, and Tensors (Chapter 8), ODEs (Chapter 9), BVP (Chapter 10) and my favourite Input/Output (Chapter 11). What it completely lacks is a chapter on Profiling (Performance Tuning) and a chapter on Debugging. That is were Wagner's book Power Programming really shines.
Finally, the Mathematica Journal is full of little gems like this: Demystifying Rules by Dr. Nancy Blachman.
My final words of wisdom is that Wolfram Language should not be one of your first computer programming languages but most definitely the last programming language you will ever need to learn. In case you care I grew up learning Basic, Pascal, Fortran 77, and C in that order before adding AWK, KSH93 and no less than half-dozen of other langauges. I tried and failed miserably to raise my kids around Forth and Scheme. Not counting TeX/LaTeX, I use only two langauges (Julia, MMA) close to 3 hours a week which is a bare minimum for maintaining languge competency.
Disclaimer: It appears that Mathematica was heavily influenced by Kenneth E. Iverson's APL beside LISP. Even though APL's central datatype, the multidimensional array, is very important in mathematics and science, I have never personally seen a single line of code written in APL or its sucessor programming language J. I could blame Fortran, MATLAB, or my birthday for it but that is the reality. While in chapter 10 (Rules-Based programming) of Computer Science with MATHEMATICA Roman E. Maeder points straight to Prolog, now forgoten logical programming language, neither Maeder nor Wagner mention APL even in passing which is astonishing considering semantic similarities with MMA. This similarity is not only superficial. When working with list data type in MAA one should be all too aware that unlike Lisp which implements list data type via linked lists, Mathematica uses arrays to implement list data type.