Chapter 4
Tracing and Debugging
When looking for bugs or trying to exploit them, it is necessary to peer inside the workings of applications. This is commonly done with the use of a debugger, such as the GNU debugger that comes with Xcode. There are some other useful tools for this purpose. One powerful feature that debuted in Leopard is DTrace, which is a kernel-level tracing API. There is also a Python interface to the debugging mechanisms in Mac OS X. Nevertheless, Apple wants some of their applications to not be traced with these mechanisms and tries to prevent this action. We’ll discuss ways around this prevention to allow tracing of even the most sensitive applications.
Pathetic ptrace
If you come from a Linux background, you may be familiar with the ptrace debugging facilities, which the Linux version of the GNU Debugger (GDB) is based on. It normally provides methods to attach and detach processes, read and write values to and from memory and registers, and offers mechanisms for program control such as single-stepping and continuing. This is not the case in Mac OS X, however.
In Mac OS X, there is indeed a ptrace() system call, but it is not fully functional. It allows for attaching and detaching a process, stepping, and continuing, but does not allow for memory or registers to be read or written. Obviously a debugger without these functions would be useless.
One other Mac OS X ptrace feature worth discussing is the PT_DENY_ATTACH ptrace request. This nonstandard request, ...
