Some generalities
I'd first like to discuss some ideas behind DSLs, and why they are useful, and then give a few pointers to some specific examples. The main idea of a DSL (at least the way I understand it) is that, for a given domain, there may be a number of primitive operations, such that all or most other desired operations can be expressed as some combinations of these. The main difference between a DSL and a simple API approach is that in DSL, you can nest these operations, and combine them in non-trivial ways. Some typical ingredients of DSL construction may involve
Parsing and creating an AST (Abstract Syntax Tree) of your code. This part may or may not be present, since in Mathematica we more or less program already in parse trees (Mathematica expressions). One desired property for a parse tree is that it (and heads out of which it is built) is completely inert, so that no piece of it evaluates. For example, in the SymbolicC case, the heads constructing the Symbolic C expressions are completely inert. This is not an absolute requirement, since you may have other means to prevent evaluation, but, at least in my experience, it is often quite hard to control evaluation when some heads in the parse tree are allowed to evaluate. I can see two kinds of situations when creating a separate parse tree (with different heads) may be needed:
- The original expression (code) of the DSL can evaluate, and it is this evaluation which actually constructs the parse tree. This may be advantageous, since the original DSL code may be (much) more compact than the resulting AST.
- The original DSL code is expressed in the "wrong degrees of freedom", which is, the operations natural for the end user of a DSL have a non-trivial mapping to the primitives, in which it is most natural to implement the desired operations.
For the (mentioned below) code formatter example, both of the above points were true.
Actual implementation of a DSL. This involves implementing the actual interactions between your DSL primitives. If your DSL will be interpreted, this is an interpreter which would execute your AST, if it will be compiled, this will be a code generator which would generate Mathematica code from your AST. Either way, this is a necessary step, which forms the essence of your DSL and defines composition of elements (language primitives).
Frequent use of recursion. Recursion is natural in this setting, because it reflects the nested nature of programs, and composition of primitive operations. I would even consider its presence necessary for a DSL to be non-trivial. To give a few examples, CCodeGenerate function from SymbolicC is deeply recursive, and also in the code formatter (mentioned below), all stages of the formatter heavily use recursion. Recursion is also central in the (also mentioned below) example from the book of Wellin et al.
Specific examples
SymbolicC and DatabaseLink are two examples of Mathematica DSLs which map Mathematica expressions to C code and SQL queries respectively. You can read their source code, it is quite instructive. In some sense, JLink can also be considered a DSL. Also in some sense, the new Statistics-related functionality forms a DSL. For more simple examples, you can look at my code formatter, which implements a simple but non-trivial formatting DSL (although I did not perform the refactoring which would make this totally apparent). Another good example is a simple graphic DSL developed in "Introduction to programming with Mathematica" by Wellin, Gaylord and Kamin (for the purposes of a pure Mathematica DSL, you can skip the parsing stage, or, more precisely, skip the lexical analysis, replacing their custom syntax with Mathematica expression-based syntax). This may actually be the easiest example to start with.
Other ways to implement DSLs
One can take a less formal route, and implement DSL-s through code-generation, achieved by writing macros. I mentioned macros in my answer on metaprogramming, and also mentioned the complications which currently accompany writing macros in Mathematica. Perhaps, the largest one is that there are no true compile and macro-expansion stages, since Mathematica is interpreted. But it should be possible to write a framework, which would introduce these, and then use it. The advantage of this method is that you can figure out your DSL's details as you go, just by eliminating boilerplate code as you see it. This may be a big advantage when you find it hard to formally specify your DSL, either because you are still learning the domain, or because it is not a priori clear what would make a good set of primitives in your DSL.
