Timing setjmp, and the Joy of Standards

July 21 2005

1. Each function returns a value which, if not NULL, signifies that an error occurred. [In other words, a simple variant of the standard UNIX mechanism, albeit one that doesn't involve the monstrosity known as errno.]
2. Using some setjmp / longjmp hackery to emulate the exceptions one finds in many modern OO languages.

setjmp / longjmp based exceptions do not suffer from the same disadvantages, and are particularly appealing when one needs to catch exceptions relatively rarely in proportion to the amount of code one has. However such a scheme is not without potential problems. One that I was worried about concerned performance, as I would need to have exception handlers in some of the most performance critical parts of my code. I could find no mention in documentation or on the web to give me a hint in this matter, so I decided to do a very simple benchmark to determine if a setjmp / longjmp based system would perform adequately.

The property I was most interested in measuring was the time taken for setjmp to run, as it would be called frequently in my proposed exception handling system, whereas longjmp would be called exceedingly rarely. The performance of longjmp is thus largely irrelevant for my purposes. I started by writing the two following programs, which would give me a very rough idea of how long setjmp takes to execute:

#include <setjmp.h>

int main()
{
  int i;
  jmp_buf env;

  for (i = 100000000; i > 0; \
    i--) {
    setjmp(env);
  }
  
  return 0;
}

#include <setjmp.h>

int f(void);

int main()
{
  int i, j;
  jmp_buf env;

  for (i = 100000000; i > 0; \
    i--) {
    f();
  }
  
  return 0;
}

int f(void)
{
  return 1;
}

The idea of these two programs is simply to time a number of 'empty' function calls (f) versus setjmp. Each program was run in the simplest possible fashion by executing gcc tx.c && time ./a.out and reading the 'user' output of time. When I ran t1 on my main OpenBSD box (a 2.8GHz P4) it took 23.73 seconds to run. When I ran t2 on the same machine it took 0.55 seconds to run. Thus setjmp appeared to take over 40 times longer than an 'empty' function call - far too slow for my needs.

At this point I decided to quickly run the test programs on another OS, just to see if the figure was about the same. So I moved to a Sun server running Solaris that I have access to (which I seem to remember has 4 processors of approximately 500MHz each). On this machine t1 ran in an astonishing 2.9 seconds, with t2 only marginally faster at a smidgen under 2 seconds. I found the results replicated on a Linux box.

At this point I was confused. Why was OpenBSD's setjmp so slow compared to other OS's? Considering my lack of regard for documentation under Linux the answer came, to my surprise, from the setjmp man page on the Linux box I had access to:

NOTES
     POSIX does not specify whether setjmp will save the signal
     context. (In SYSV  it  will  not. In BSD4.3 it will, and
     there is a function _setjmp that will not.)  If you want
     to save signal masks, use sigsetjmp.

In other words, OpenBSD's setjmp was actually doing a lot more work than Linux or Solaris, since it was saving the signal mask. Worse this discrepancy between operating systems appears to be perfectly legitimate in POSIX! Returning to OpenBSD's setjmp man page, the following suddenly made a lot more sense:

CAVEATS
     Historically, on AT&T System V UNIX, the setjmp()/longjmp()
     functions have been equivalent to the BSD _setjmp()/
     _longjmp() functions and do not restore the signal mask. 
     Because of this discrepancy, the sigsetjmp()/siglongjmp()
     interfaces should be used if portability is desired.

While I will admit that I am surprised at the huge amount of extra time saving the signal mask takes, I was happy to uncover the cause of the discrepancy. I was particularly happy due to the fact that my program does not need to save or restore the signal mask; thus I could happily sidestep this lengthy operation without penalty.

Rewriting t1 to the following:

#include <setjmp.h>

int main()
{
    int i;
    jmp_buf env;

    for (i = 100000000; i > 0; i--) {
        sigsetjmp(env, 0);
    }
    
    return 0;
}

There are two lessons I've taken away from this little exercise, both of which I think we all know, but of which I - at least - need reminding of from time to time. Firstly, you can never be sure of anything until you measure it - I would never have guessed that saving and restoring the signal mask was such a time-consuming operation, for example. Secondly, standards don't always determine every aspect of an implementations behaviour.

Comments