Great article, thanks! Tiny suggestion: change "every non-empty sequence of lower-case digits defines an integer" into "every non-empty sequence of digits defines an integer".

@Gonzalo Oops, that's an amusing typo -- fixed!

The regular expressions are closed under intersection and complement (unlike the CFLs), so in the case of wanting to exclude "pi", it seems as if you'd actually want to keep a separate scanner, and just extend lex's syntax to allow you to write

ident = [A-Za-z]+ - "pi" - "e" - ...

where the + is the non-zero Kleene closure operator and the - is subtracting out possibilities. Of course, the point from your great interview with Eelco about wanting to compose grammars is valid too, so it feels as if we're being pulled in two directions at once...

@Michael Your "-" operator suggestion is an intriguing one and would have made the "pi" example easier to express, once I'd understood where the overlap was. My initial reaction is that I think "-" should work in lex, because it restricts itself to "true" regular languages, and therefore in my imaginary lecc too.

That said, you're very right that once you start composing grammars (scannerless or otherwise), bigger challenges start to make themselves known. Parsing is full of fun trade-offs!

For anyone else wondering about the "partial fix" (the link is broken): "Let's Accept Rejects, But Only After Repairs" by Jan van Eijck (see here or pages 117-128 of this book).

@Julian Thanks for the 404 notification. The first link doesn't show anything for me, but the second leads to a PDF. I've put this in a footnote in the post -- thanks!

@Laurence Sorry to bother you again: I believe something went wrong with the link in the footnote, the PDF does not actually contain the paper.

@Julian Huh. I can only assume I copied the wrong link (I'm hoping the content doesn't change over time!). I've found another link which seems to download the right thing for me. Fingers crossed!