Subclassing and Hooking with Visual Basic

Some WH_DEBUG Scenarios

In this section, we’ll examine some common problems that arise when debugging hooks, and show how you can use the WH_DEBUG hook to diagnose and solve them.

Problem: Misbehaving System-Wide Hook

Sometimes a misbehaving system-wide hook can interfere with your hook’s operation. Consider the situation in which your application installs a system-wide keyboard hook when the system first boots up. Later on, a third-party application installs a keyboard hook. This keyboard hook, however, does not call the CallNextHookEx function at the end of its filter function, as a well-behaved hook should. When your application is running, you notice that your hook does not work, or worse, that it works intermittently.

To determine the cause, you can install a WH_DEBUG hook on the same thread as the target hook, and only watch for calls to any keyboard hooks. Watching for the WH_KEYBOARD constant in the wParam parameter of the WH_DEBUG hook allows you to watch calls that are made to the WH_KEYBOARD hook’s filter function. This WH_DEBUG hook will intercept hooks installed with system-wide scope, or those installed from the same thread as the WH_DEBUG hook.

After you install the WH_DEBUG hook, you notice that it is capturing information about a hook that has the same idThread as the thread that contains your WH_DEBUG hook as well as your keyboard hook. But when you inspect the idThreadInstaller member, you notice that it is different from the idThread value. This indicates that ...