Secure Programming Cookbook for C and C++

12.13. Detecting Unix Debuggers

Problem

You need to prevent someone from debugging a Unix binary.

Solution

Single-stepping through code in a Unix environment causes a SIGTRAP to be sent to the process. The debugger captures this signal and allows the user to examine the state of the process before continuing execution. By installing a SIGTRAP handler and sending itself a SIGTRAP, the process can determine whether it is being debugged.

Discussion

The spc_trap_detect( ) function is used to install a signal handler to catch trap signals sent to the target, then issue a trap signal. The SPC_DEBUGGER_PRESENT macro checks the num_traps counter managed by the trap signal handler; if the counter is zero, a debugger is capturing the trap signals and is not sending them to the process.

#include <stdio.h>
#include <signal.h>
   
#define SPC_DEBUGGER_PRESENT (num_traps =  = 0)
static int num_traps = 0;
   
static void dbg_trap(int signo) {
  num_traps++;
}
 
int spc_trap_detect(void) {
  if (signal(SIGTRAP, dbg_trap) =  = SIG_ERR) return 0;
  raise(SIGTRAP);
  return 1;
}

The following example demonstrates the use of spc_trap_detect( ) to initialize the debugger detection, and SPC_DEBUGGER_PRESENT to check for the presence of a debugger:

int main(int argc, char *argv[  ]) {
  int x;
   
  spc_trap_detect(  );
  for (x = 0; x < 10; x++) {
    if (SPC_DEBUGGER_PRESENT) printf("being debugged!\n");
    else printf("y\n");
  }      
  return(0);
}

This detection method is not particularly effective because most Unix debuggers allow the trap signal ...