It’s just over a year since I first got my grubby little mitts on a machine with a dual-core processor (the fact that it took me over 12 months to get the rest of the machine fully working with my other equipment is annoying, but irrelevant to this entry). Such processors are gradually finding their way into more and more peoples hands. The era of widespread multi-processor machines is finally upon us - albeit rather later than many of us expected, and in the form of multicores rather than multi-processors.
The advent of such machines is having an odd effect on many in the software community, which is having what amounts to a collective crisis of confidence in its ability to fully utilise such machines. But before exploring this further, let’s take a step back in time.
The software crisis
I remember as an undergraduate being told - in Dickensian tones that suggested This Is The Way It Always Has Been And Always Will Be - that we were in the midst of a software crisis. Software was always behind schedule, over budget, unreliable, lacking features, and generally unusable. To a large extent I bought into this self-flagellating view of the world - after all, I saw software crash all the time.
Then a few years back, I was at an OMG dinner (in Disneyland in California, but that’s a detail I’d prefer to forget), and among the 10 or so people there, the topic of conversation shifted onto the software crisis. Various people shook their fists and banged the table - metaphorically speaking - decrying the terrible state of software. Then Jim Rumbaugh - of UML fame, and who retired from IBM / Rational last year - said something that I initially dismissed, but realised some time later was incredibly profound. To paraphrase Jim: “We’ve forgotten how incredible software is today. I loaded Photoshop onto my computer last week and within minutes I was manipulating photos in ways that would have been impossible for even world experts a few years ago. And that photo editing requires a reliance on many large, complex software systems that have been packaged in a way that my family of non-experts can fairly easily use.”
What Jim was saying changed my thinking on software. While software could certainly be better, couldn’t everything in life? The fact that things can be improved doesn’t mean that the current state of affairs is intolerable in an absolute sense. In actual fact, for normal people the software that they interact with is pretty decent these days. It may not be perfectly reliable, but it’s generally more than reliable enough (gone are the days where machines need to be rebooted 5 times a day to keep them stable). It may not be as easy to use as it could be, but normal people manage to do most of the tasks they need to without huge problems and that’s the ultimate acid test.
A substantial reason why we worked our way out of the software crisis is that we have become much better at developing software over the last 20 years. We have better languages and tools; we understand many of the problems more thoroughly; we have been able to disseminate knowledge about software creation fair and wide; and we have become much better at reusing the increasingly large number of mature, stable programs available.
While one still does hear people witter on about the software crisis from time to time, this is gradually diminishing because the reality of software today is that it’s largely fit for purpose.
Is there a multicore software crisis?
I have a feeling that software people secretly miss the opportunity to whinge about the software crisis. But somethings coming along that may replace it. Put simply: “Now we’ve got these multicore processors, and we can’t fully utilise all that power, all our development methods are dead.” Cue throwing toys out of prams etc. While I understand the reasoning behind this mode of thought, I largely disagree with it. Here’s why.
When I was talking about the software crisis earlier, I missed out the other factor in the demise of said crisis: hardware. Todays computers are so incredibly fast, so capable of dealing with mind-boggling quantities of data, that they bear little relation to those of 25 years ago. The huge increase in horsepower has made many software practises that were previously untenable - e.g. using dynamically typed languages like Python - more than useable. Consequently we have been able to develop software with increasingly little concern of the underlying hardware. In my opinion, reasonably priced PCs are more than fast enough for virtually every task that normal people throw at them, and this has been the case for the last 5 or 6 years. Hardware speed increases since that point have been largely lost on your average computer user, because the machines were already fast enough.
The last sentence of that last paragraph is absolutely key for me: for the vast majority of tasks, for the vast majority of users, machines are already fast enough. Of course we’ll take more speed if it’s given to us, but the lack of speed isn’t generally holding us back any more.
So if you’re developing a desktop application, or a standard web site, or a back-end processing system, the chances are that your development tools, languages, and methods are actually largely adequate. More accurately, the next generation of tools, languages, and methods will probably be a useful - but not radical - evolution of the current generation.
Why then are many people preaching that we need to rip all of our tools, languages, and methods to shreds and start again?
Are there situations where we need to rethink software for multicores?
I rather enjoy having a multicore desktop machine as it enables me to work quite a bit faster than before. The reason for that is - and I’m man enough to admit it - I’m not normal. When I use a computer, I’ve often got a long-running CPU bound task going on, or I’m switching rapidly between different applications. Because all these things run as different processes, my OS is able to share a reasonable amount of the grunt work between cores, thus ensuring that my machine remains responsive. The fact of the matter is that only a tiny minority of people will ever run their machines in such a way - and for those of us that do, the technology for distributing processes across cores is already more than adequate. As this suggests, the vast majority of todays software is more than adequate for the multicore world as users will perceive it.
I do however think that there are certain classes of problems that could benefit substantially from multicores, but which are not efficiently decomposed into coarse-grained processes. The most compelling for me is computationally intensive scientific applications. Some of these applications crunch numbers like there’s no tomorrow, and often have aspects which are highly parallelizable at the fine-grained level. It might also be that some computer games could benefit similarly (given that many of them are number crunchers aimed at the entertainment domain), but frankly I’m so out of touch with that area that I don’t feel qualified to offer an opinion.
My fundamental points here are: most people won’t notice the difference in speed from multicores; for those of us who do benefit, coarse-grained process decomposition pushes utilisation more than high enough; and only a few very specialised domains will really benefit from multicores.
How do we best utilise multicores?
For most cases, the answer to me is clear: breaking systems up into a small number of cooperating processes is more than sufficient. In a small number of cases, threads might be an answer, but threads are too often a ticking time-bomb (my prediction is that multicore processors will highlight huge numbers of timing problems in existing multi-threaded applications). Existing languages, tools, and methods, are perfectly suited to the former, and are sometimes adequate for the latter.
For those rare, specialised domains such as computationally intensive scientific applications I think a new approach is needed. I say this not to advocate change for changes sake, but because I believe that the developers of such applications are unusual in that they are prepared to absorb a large amount of implementation pain if they can significantly improve their execution time.
Step forward functional languages without mutable state.
I often enjoy teasing the FP community. Frankly FP has never shown itself as being a practical way to develop most systems: cute 10 line programs simply don’t reflect the ugly realities of changing requirements and developers of varying abilities. But FP without mutable state has two inherent advantages over imperative approaches. First, it can largely do away locks, since locks are generally only needed to protect mutable state. Second, functional programs are often amenable to more analysis than imperative programs, and consequently parallelization optimisations are more likely to be identified automatically.
I think that this is an area where FP could finally find its niche. The path has been somewhat mapped out by Erlang, but Erlang is an outlier in that, while it’s an FP language without mutable state, it doesn’t have static types. Every other FP language without mutable state that I know of has a static type system. Most such type systems are merely odd on a good day, but wilfully obscure on a bad day - they get in the way too often. If someone can come up with a statically typed language which is as relatively easy to use as Erlang, but can provide large parallelization benefits, then there will be a specific class of real user out there who will gobble it up.
Conclusions
My main contention is that multicore machines aren’t really going to make a big impact on most people or most developers. The vast majority of software will continue to be developed using methods that are familiar in tone to todays developers, and the resulting software will be sufficiently efficient, feature rich, and stable. However for a small class of users, existing techniques are lacking. In todays software world, which is more tolerant of heterogeneous systems than ever before, this provides a niche opening for FP if it can be packaged in a reasonable fashion.
I do want to make clear though that I don’t think that FP in any of its forms will ever take over the world, but I’m going to be interested to see if FP finally manages to carve out a distinct niche.