More Meta Matters

August 30 2006

Here is the problem I was facing in a nutshell. In the new version of the Converge language I was making much more use of meta-classes to hide the general ickiness of primitive datatypes. A simple example is the File class. When one opens a file for reading or writing, the resultant object needs to have extra space in memory to record the C-level file handle. Other types of objects may similarly need to have an arbitrary extra space to record some low-level information. Many languages attempt to hide this fact altogether, but that tends to rule out some useful things, and makes the language hard to extent. Far better in this case to have a Meta_File class which specifies how new File objects are to be created; users can also add their own meta-classes to create arbitrary types of objects as they see fit.

The Meta_File class (it isn't actually called that in Converge, but this name makes things easier to explain in this entry) is in C and its definition looks like this in hybrid Converge and C code:

class Meta_File(Class):
  func new(path, mode):
    Con_Obj *new_file_object = malloc(sizeof(normal object) +
      sizeof(extra space for file handle));
    ...

class File metaclass Meta_File:
  func read(num_bytes):
    FILE *handle = (((u_char *) self) + sizeof(normal object))->
      handle;
    ...

File := Meta_File.new("File", [Object], [func new(): ...])

class Read_Line_File(File):
  func readline():
    ...

The fix for this is obvious enough: the Read_Line_File class needs to declare that its metaclass is Meta_File:

class Read_Line_File(File) metaclass Meta_File:
  func readline():
    ...

So I set about devising a mechanism which would ensure that subclasses would, by default, use the same metaclass as their superclass whilst still being ObjVLisp in spirit (Python's __metaclass__ attribute, for example, is decidedly non-ObjVLisp in style). Eventually I came up with a mechanism that I was happy with, and having used it for a while, decided that it was worthy of being recorded in a research paper.

I then started hunting around for all the past work I could find on metaclasses, stumbling along the way on papers that I had seen at some point in the past, but had not been able to digest. Most of what I came across had nothing to say about the issue above, but I saw a couple of references to a concept called Metaclass Compatability. Simplified somewhat, this refers to the potential problem when a language with multiple inheritance inherits from two superclasses which have different metaclasses. An interesting read is from Nicolas Graube (or the similar Bouraqadi-Saâdani et. al., also available via CiteSeer). The problem of metaclass compatability is, frankly speaking, a largely theoretical problem - it's a corner case which I struggle to believe is likely to occur often (if at all) in practise, and the treatment of it is really rather dense.

What's interesting about the metaclass compatability research is that, once one strips away the theoretical densenes, one discovers that the solutions that are proposed for metaclass compatability effectively present a solution to the problem I outlined earlier in this entry - and which are very similar to what I eventually came up with. My first feeling was of disappointment that I hadn't discovered something novel. My second feeling was also of disappointment: if I'd been able to interpret this research up front, I might have saved myself a lot of effort. But ultimately I realised that I'd merely been a victim of the fundamental problem whenever one discusses theory and practise: even when each route leads to the same answer, that route is often impenetrable to the other side until both independently arrive at the same answer.

Link to this entry