Debugging A Failing Hotkey

On Sunday I tried using hk, but it wasn’t working. That surprised me, because hk is a very simple program that allows one to set temporary hotkeys — and a program I use many times a day. Here’s a toy example of hk:

hk Ctrl+Shift+F8 echo Hello

hk waits until Ctrl+Shift+F8 is pressed — when it is, hk executes the command (which prints “hello”) and then exits.

I use hk in several ways, but what motivated me to write is because it allows me to use xdotool to enter passwords and the like without having to copy them to the clipboard. For example, I use oathtool to generate a time-based one-time password, pipe the output into hk, go to my browser and press Ctrl+Shift+F8 to have xdotool type type the one-time password in on my behalf:

oathtool -b --totp=SHA1 abcdefgh \
  | hk -w Ctrl+Shift+F8 xdotool type --file -

where abcdefgh is the secret you need for the place you’re generating the one-time password for. Here’s an example of hk in action (you’ll have to trust me that I press Ctrl+Shift+F8 a few seconds into the video):

But, on this day, hk didn’t want to execute xdotool. No matter how often, or how hard, I pressed the hotkey I had assigned, nothing was happening. I went through my standard list of Unix problems. I hadn’t hit the limit of processes, threads, or open files. Executing xdotool on its own worked fine. I could pipe data to other programs successfully.

At this point I realised that hk was almost certainly the culprit, and loaded up the source code. The good thing about hk is that it’s under 300 lines of code. The bad thing about hk is that it’s written for X11, which – and I don’t know a nice way of saying this – has a weird, complex, model of keyboards and key presses. I wasn’t looking forward to reminding myself of that API and finding out what mistakes I might have made in trying to use it.

After putting in a few printfs, I soon realised that the problem was occurring due to hk’s -w “wait” feature. By default, hk executes a command as soon as the hotkey is pressed — even if some or all of the hotkeys remain pressed. This is mostly fine, except if the command you’re executing does different things depending on what keys are pressed.

xdotool type does just that. If, for example, “shift” is held when it’s typing, letters will be capitalised; if “ctrl” is held, then you can find your program doing all sorts of odd things. It’s quite difficult to press and immediately release all keys, so at first hk and xdotool type didn’t play together very well. After experimenting with fixed pauses, I quickly realised that hk needed an extra feature to handle this, so -w informs hk that, after the hotkey has been pressed, it should wait until there are no keys pressed [1] before executing the command.

There is, however, a slight catch: in X11 some keys are “pressed” even though the user isn’t actively pressing them. For example, if you have “caps lock” on, then X11 considers that key to be “pressed”, which means that hk -w seems to stall indefinitely. hk -w therefore ignores the following keys even if X11 says they’re “pressed”: caps lock, num lock, and scroll lock.

The challenge with this approach is that I have no idea what other keys X11 might say are “pressed” when I as a human wouldn’t consider them to be. This isn’t a theoretical worry. When I bought a Framework laptop, I found that (at least on OpenBSD) a key called “mode switch” [2] is permanently pressed — so hk ignores that too. However, since nothing about my hardware or software had changed since hk had last worked, it seemed unlikely that one of these sorts of keys could be responsible. I loaded up xev to see if there were any clues, but there weren’t.

I thus inserted a few lines in hk to print out the keys that were causing hk -w to keep waiting. Apparently 0xff97 was being pressed. I quickly executed the following command to see what that key was:

$ rg 0xff97 /usr/X11/include
X11/keysymdef.h
213:#define XK_KP_Up                         0xff97

I know enough about X11 to know that must be the “keypad up” key. That surprised me because the keypad on my keyboard is miles away from where my fingers were: I couldn’t possibly be pressing the 8/up key by accident.

One thing I’ve learnt about programming is that nearly every mistake I observe is my own fault. Yet, as I looked at the code, I couldn’t see anything which might plausibly be causing hk to claim that “keypad up” (or “8”) was being pressed. I was baffled.

After a few minutes, something occurred to me. It was Sunday. On Saturday I had done a rare deep clean of my desk — including my keyboard. If you’ve never cleaned a keyboard out, you might be astonished at the amount of dirt and dust that hides between keys. I’d done my normal “turn the keyboard upside down and bang it a few times” to get the dirt out, then used a very slightly damp cloth between keys to get most of the dust between keys out.

I quickly realised that I’d turned “num lock” off, but hk still wasn’t working. Becoming increasingly suspicious, I looked at the “keypad up” key. From a top-down view everything looked normal. But then I rotated the keyboard a little bit, and the problem was obvious. The key was slightly, though noticeably, lower than other keys — in other words, it had got stuck in the “pressed” position! I gave it a light tap, the key popped back up to its normal position, and hk -w started working as expected.

So, yes, I’d just spent 30 minutes debugging a literally stuck key on my keyboard. Since I’m probably not going to be the last person to be caught out by this, whether it’s due to an X11 oddity or overzealous cleaning, I’ve added a -v option to hk. When a hotkey is pressed and -w is used, -v periodically (currently once a second) prints out which keys remain held. Here’s an example of me pressing the hotkey (Ctrl+Shift+F8) and then only slowly taking my fingers off Ctrl and Shift:

I’ve learnt a couple of lessons from this.

First, continually pressed keys don’t behave in X11 quite as I expected. If I press and hold “up”, it repeats in the way I expect, but if I then press and release another key (say “a”) while still holding down “up”, then “up” stops repeating — but is still considered to be pressed. That’s how I managed to use my computer for an hour without realising that “up” was considered pressed.

Second, cleaning my keyboard might save me from incubating new forms of life, but it can cause other problems.

Still, at least I now know that hk wasn’t to blame — and hk-0.3.1 should help me if this issue ever crops up again!

Footnotes

Comments