Chapter 6

Reverse Engineering

In earlier chapters you learned how to peer inside a running process on Mac OS X to see what is happening. This involved using a couple of dynamic-analysis tools. In this chapter, you will continue to investigate the inner workings of Mac OS X binaries, this time by looking at the static disassembly of Mach-O binaries. To this end, we’ll show you some techniques to help clean up some of the most common problems that IDA Pro encounters with this file format. We will then discuss some particulars of disassembling binaries originating from Objective-C (Obj-C). Finally, we’ll walk you through an analysis of a binary and illustrate how you can change the core functionality of binaries rather easily once you understand how they work.

Disassembly Oddities

When looking at Mac OS X x86 binaries in IDA Pro that don’t come from Objective-C code, you realize that they look pretty much like binaries from other operating systems. Objective-C binaries look quite a bit different, and we’ll describe those later in this chapter. You’ll run into a few issues for which IDA Pro fails to provide optimum disassembly. We discuss these as well.

EIP-Relative Data Addressing

One unusual construct you’ll notice when disassembling Mac OS X binaries typically occurs at the beginning of each function. You’ll see that data is often referred to neither globally nor as an offset from the beginning of the function, but from some other point, which we’ll call an anchor point; see

Get The Mac Hacker's Handbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.