Stub mechanics and the userland exec

First, it is necessary to understand that a software protector is actually made up of two programs:

  • Protection phase code: The program that applies the protection to the target binary
  • Runtime engine or stub: The program that is merged with the target binary that is responsible for deobfuscation and anti-debugging at runtime

The protector program can vary greatly depending on the types of protection that are being applied to the target binary. Whatever type of protection is being applied to the target binary must be understood by the runtime code. The runtime code (or stub) must know how to decrypt or deobfuscate the binary that it is merged with. In most cases of software protection, there is a relatively simple ...

Get Learning Linux Binary Analysis now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.