General Purpose Programming Languages' Speed of Light

Having recently returned from a week of talks on programming languages, I found myself wondering where general purpose programming languages might go in the future; soon I wondered where they could go. The plain truth about programming languages is that while there have been many small gains in the last 20 years, there have been no major advances. There have been no new paradigms, though some previously obscure paradigms have grown in popularity; I’m not even aware of major new language features (beyond some aspects of static type systems), though different languages combine features in slightly different ways.

The lack of major advances in a given time period may not seem surprising. After all, major advances are like earthquakes: they occur at unpredictable intervals, with little prior warning before their emergence. Perhaps we are on the cusp of a new discovery and none of us – except, possibly, its authors – are aware of that. That is possible, but I am no longer sure that it’s likely. If it doesn’t happen, then it seems clear to me that we are getting ever closer to reaching general purpose programming language’s speed of light—the point beyond which they cannot go.

Programming languages cannot grow in size forever

The basis of my argument is simple:

1. Every concept in a programming language imposes a certain cognitive cost upon users.

2. Our brains ability to comprehend new concepts in a language is inversely proportional to the number of concepts already present.

3. At a certain number of concepts, we simply can’t understand new ones.

Note that I deliberately phrased the first point to reflect the fact that some concepts are more complex to learn than others; but that, at a certain point, the volume of concepts becomes at least as important as the cumulative complexity because of the inevitable interactions between features. In other words, if language L1 has 20 features, of which 6 are complex and 14 simple, the cognitive cost is similar to L2 which has 3 complex and 17 simple features.

This doesn’t sound too bad, until one considers the following:

1. The core of virtually every extant programming language is largely similar.

2. That core contains many concepts.

3. Many of those concepts have a high cognitive cost.

In other words, most programming languages are surprisingly similar at their core. Before we can differentiate L1 from L2, we must already have understood features ranging from variables to recursion, from built-in datatypes to evaluation order.

Programming languages’ core

The overly earnest teenage boy that lives within all of us wants to believe that programming languages are massively different. At any given moment, someone with a nihilistic bent and an internet connection can find fans of different programming languages fighting pitched battles where language L1 is argued to be always better than L2. The dreary irony, of course, is that what L1 and L2 share in common is nearly always greater than that which differentiates them. How many people use programming languages without function calls? without recursion? without variables? without …? Nobody does, really. Those languages that are truly unique, such as stack-based languages, remain locked in obscurity.

The reality is that, since the late 1950s, programming languages have steadily been converging towards a stable core of features. That core has proven itself sufficiently good to allow us to develop huge pieces of software. However, it leaves our brain surprisingly little capacity to understand new features. Try and do too much in one language, and problems accrue. We long ago developed the technical ability to create programming languages too complicated for any single person to fully understand. C++ is the standard (though not the only) example of this, and far be it for me not to pick on an easy target. As far as I can tell, not a single person on the planet understands every aspect of C++, not even Stroustrup. This is problematic because one always lives in fear of looking at a C++ program and finding uses of the language one doesn’t understand. At best, individuals have to continually look up information to confirm what’s going on, which limits productivity. At worse, people misinterpret programs due to gaps in their understanding; this introduces new bugs and allows old bugs to hide in plain view.

In an ideal world, we would be able to understand programming languages in a modular fashion. In other words, if I don’t use a feature X, I don’t need to know anything about X. Sometimes this works, and organisational coding standards are one way to try and take advantage of this. If this modularity property held fully, it would allow us to add as many modularised features to programming languages as we wanted. However, programming rarely involves creating a program in isolation from the rest of the world, particularly in the modern era. Instead, we pull in libraries from multiple external sources. Even if I don’t want to use feature X, a library I use might; and, if it does, it can easily do so in a way which forces me to know about X. Take C++ again: I might not want to use templates in my C++ programs but most modern C++ libraries do. Even if a programming language is designed so that its features are modular, the social use of such a language tends to destroy its modularity. This explains why it’s virtually impossible to be a competent programmer without a reasonable understanding of nearly every feature in a language.

Better education?

A reasonable question to ask is whether this ability to exceed the human brain’s capacity is a temporary one, due to poor education. As time goes on, we tend to become better at condensing and passing on knowledge. I have heard it said, for example, that what might have baffled an early 20th century physics PhD student is now considered a basic part of an undergraduate education (since I struggled with physics as soon as it became semi-mathematical, I am not in a position to judge). Perhaps, given time, we can teach people the same number of concepts that they know now, but at a lower cognitive cost.

I do expect some movement along these lines, but I’m not convinced it will make a fundamental difference. It seems unlikely that programming students of the future will find it substantially easier to learn recursion or iteration, to pick two semi-random examples. Every language needs to choose at least one of these concepts. Of the two, most people find recursion harder to grasp, but nested iteration doesn’t come naturally to many people either. Those of us who’ve been programming for many years consistently underestimate how difficult newcomers find it. Starting programming education from a young age might change this dynamic, but it’s difficult to imagine that sufficient quantities of competent programmers will move to become teachers, at least in the short term.

Better language design

Let’s assume, briefly, that you agree that C++ is too complex but are wondering whether its difficulties are due to bad design rather than size. Perhaps all I’ve really done so far is point out that badly designed languages overload our brains sooner than well designed ones? There is clearly some truth in this but, again, I’m not sure that it fundamentally changes the landscape.

Let’s take two statically typed functional languages as an example of more considered language design: Scala and Haskell. Both languages aim to stop users shooting themselves in the foot through the use of expressive type systems. Maybe by constraining users, we can help our brains absorb more general language concepts? Alas, I am not convinced. Some of the things Haskell and Scala’s type systems can express are astonishing; but I have also seen each type system baffle world-renowned experts. Type systems, like any other language feature, do not come for free. Even if we could design a perfect programming language, I doubt it could be much larger than such languages are today.

What the above means is that the space for a major advance in general purpose programming languages is limited. The analogy with the speed of light works well here too. As an object’s speed approaches the speed of light, the energy required to accelerate it reaches infinity, which is why a normal object can’t ever travel at the speed of light. Worse, the energy required is non-linear, so the closer you get to the speed of light, constant increases in energy lead to ever-slower acceleration. Language design is rather like that. Beyond a certain point, every feature – no matter how well designed – has a disproportionate cost. It takes longer to understand, longer to learn how to use idiomatically, and longer to understand its interactions with other features.

Tooling

The tooling that surrounds programming languages has changed significantly in the last two decades. Most developers now use huge IDEs [1], which offer a variety of ways to understand and manipulate programs. Verification techniques and tools have made major advances [2]. And whole new styles of tools now exist. To take two examples: Valgrind allows one to analyse dangerous run-time behaviour in a way that can significantly tame the danger of even unsafe languages such as C; Moose allows one to gain a reasonable high-level understanding of large programs in a fraction of the time it takes to wade through source code by hand.

Clearly, such tooling will only increase in quantity and quality as we move forwards. But will it help us understand programming languages more effectively? Again, I expect it to help a bit, but not to make a fundamental difference. Perhaps future editors will have the ability to simplify code until we zoom in (rather like an extreme version of folding). Ultimately, however, I don’t know how we can avoid understanding the detail of programming languages at some point in the development process.

Moving beyond general purpose

What are we to do? Give up on general purpose programming language design? No—for two reasons. First, because a degree of group-think in language design means that we haven’t explored the language design space as well as we should have yet. For all we know, there could be useful discoveries to be made in unexplored areas [3]. Second, because the general purpose part of the name is subtly misleading. A traditional assumption is that every programming language has been expected to be applicable to every problem. This then leads to the my language is better than yours notion, which is clearly nonsense. Anyone who tells you their favoured programming language – or even their favoured paradigm – is always the best is delusional. There is no such thing, and my experience is that programming language designers are almost universally modest and realistic about the areas to which their language is best suited.

In a single day, I can do Unix daemon programming in C, virtual machine development in RPython, system administration in the Unix shell, data manipulation in Python, and websites in a hodge podge of languages. Each has strengths and weaknesses which make it better suited to some situations than others. People who need certain styles of concurrent programming might favour Erlang for that, but more conventional languages for sequential processing. Many use cases are currently uncatered for: languages which target power efficient execution may become of greater interest in the future. All in all, I suspect we will see a greater level of customisation of nominally general purpose languages in the future.

I also suspect we will see greater use of more obviously restricted languages, which are often called domain specific languages [4]. Rather than try and make features that work well for all users – and that must still be understood by those for whom they don’t work well – we are likely to have better luck by focussing on specific groups and making their life easier. For example, I do not wish to have my programming language cluttered with syntax for traditional mathematics (beyond the bare essentials of addition and the like), because I don’t use it often enough. A programmer crunching data to produce statistics, on the other hand, would be much more productive with such a syntax. My current guess is that we will build such languages by composing smaller ones, but there are other possible routes.

In many ways, neither of these outcomes is quite as exciting as the notion of a perfect language. The contemporary emergence of a wide class of acceptable general purpose languages is an acknowledgement that we can’t squeeze every feature we might like into a single language—our brains simply can’t handle the result. Rather than try and move general purpose languages beyond the speed of light, we’ll probably end up with many different points near it. At the same time, the possibility of restricted domain specific languages may enhance our productivity in narrower domains. This is, perhaps, not as immediately exciting a prospect as targeting all users, but it is its relative modesty that makes it more likely to pay off. Furthermore, unlike general purpose languages, that journey is nearer its beginning than its end. We might be reaching the speed of light for general purpose programming languages, but we’ve barely started with domain specific languages.

And, of course, there’s always the remote possibility of an unforeseeable earthquake hitting programming languages. Unlike in the real world, we are not callous or inhuman for hoping that such an earthquake will hit us, even though many of us are not sure it will ever come.

Acknowledgements: My thanks to Martin Berger, Carl Friedrich Bolz, Lukas Diekmann, and Naveneetha Vasudevan for insightful comments on an early draft of this article. All opinions, errors, and infelicities are my own.

Footnotes

Comments