Many of us are used to being told that macros are useful. However few of us really understand why. A large-ish, if gradually dwindling, number of people use C, and its preprocessor, on a regular basis. This gives them access to a crude, dangerous, but surprisingly effective, macro system. However the type of macro system we are told is most useful is that of a LISP-like language. Commentaries, such as this article extolling the virtues of Scheme-like macros, continually reinforce the notion that macros are a programming nirvana. Claims of an order of magnitude improvement in developer time when macros are used are not uncommon. And because most of do not, and probably have never, used a language with a real macro system, we tend to be somewhat in awe of the programming intelligentsia who broadcast such messages.
For quite some time I have held a somewhat different opinion. For modern - and I use that word very deliberately - programming languages, I believe that LISP-like macros in their raw form aren’t hugely useful. Some justification for this position is thus in order.
I designed the Converge language which is one of the few modern programming languages with a macro system. Because its macro system is fairly directly inherited from Template Haskell, it’s rather more verbosely referred to as a compile-time meta-programming facility; but really, it’s just a macro system. From a practical point of view, there is only one substantive difference between Scheme-like macros and Converge macros. Scheme explicitly identifies macros, and then function calls which reference that macro magically turn into macro calls. In Converge, macros are just normal every-day functions, but the call site of the macro is explicitly identified. From an expressivity perspective, the two approaches can be considered equivalent.
Adding a macro system into Converge was no small task. I had to understand a lot of things that my lazy side would rather have glossed over and I had to make innumerable mistakes before I got to a reasonable design. Fairly early on in this process I realised that there were only ever likely to be a few normal Converge programs that were likely to benefit from raw LISP-like macros. To see why, we need to take another step back.
Compared to Converge, LISP in its purest form is almost unimaginably spartan. In fact, most of the successful programming languages that date from around the early 70’s or earlier, tend to lack features that most programmers now take for granted (although at least LISP and its descendants feature automatic memory management). As a general rule I am all for simplicity in life, being something of a simpleton myself. However simplicity is not an end in itself. Stone benches have an integrity, and air of stability about them, that no sofa can match; but I’ve not been to many houses with a stone chair in the front room. Thus an inevitable side effect of spartan languages is that people need to encode extra functionality in order to make life somewhat more bearable. Macros are an incredibly powerful way to encode such functionality in LISP-like languages. For example, you want an object orientated style system on top of LISP? Use macros. Thus macros are an integral part of the modern LISP experience: they allow users to raise the level of abstraction of the programming language.
The reason why raw macros are not especially useful for most Converge programs is that the base language itself is fairly feature rich. This is one reason why I used the word modern earlier. For example, no one in their right mind is likely to use macros to create an OO layer in Converge; it already has a perfectly serviceable one. In fact, for the majority of uses of macros in LISP, the chances are that it’s not worth the effort to create an analogue in Converge. The LISP community has traditionally thought that raw macros raise the level of abstraction of any programming language they’re inserted to; in other words, they raise the level of abstraction relative to its starting point. My experience on the other hand is that raw macros instead raise the level of abstraction to an absolute level. Put crudely, if macros raise the abstraction level to X, and your language is abstraction level X-1, then macros will be a gain; but if your language is already at abstraction level X you’re not going to notice much improvement.
Assuming you agree with me that raw macros aren’t hugely useful for modern programming languages, you might reasonably ask: why did you continue implementing such a thing in Converge? Here we see why I’ve used the term raw macros earlier. Converge has raw macros because they are the lowest common denominator of compile-time meta-programming (and thus far the shipping Converge system uses precisely one raw macro call, and it’s not a particularly crucial one). However Converge also contains a second feature, the DSL block which is a simple layer on top of raw macros which allows arbitrary syntaxes to be neatly embedded in a Converge file and compiled out, while still retaining excellent debugging support.
It’s too early to state with confidence whether DSL blocks are a successful or practical means of improving the level of abstraction of Converge. However it does give some insight into the question posed at the beginning of this article. Macros are useful when they give the user the ability to rise above the base programming language. Thus raw macros are a boon to LISP, but offer little to Converge. DSL blocks seem to confer an advantage to Converge, but the programming languages of the future may subsume such functionality.
I do not think that there is a fundamental law of the programming universe which says macros always increase the level of abstraction. Macros aren’t an end in themselves. If programming languages incorporate macros in such a way that they help users raise the level of abstraction, then they are useful. The way in which macros achieve that will evolve as programming languages evolve. And if macro technology fails to keep up, or proves inadequate for the job, then macros will no longer be useful. Already I think that LISP-style raw macros are gently heading towards obscurity. Perhaps languages such as Converge and Metalua, as immature as they currently are, will point to a new chapter in macro technology and dissemination.