Mastering Reverse Engineering

Book Description

Implement reverse engineering techniques to analyze software, exploit software targets, and defend against security threats like malware and viruses.

Key Features

  • Analyze and improvise software and hardware with real-world examples
  • Learn advanced debugging and patching techniques with tools such as IDA Pro, x86dbg, and Radare2.
  • Explore modern security techniques to identify, exploit, and avoid cyber threats

Book Description

If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices.In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code.

By the end of this book, you will have the confidence to perform reverse engineering.

What you will learn

  • Learn core reverse engineering
  • Identify and extract malware components
  • Explore the tools used for reverse engineering
  • Run programs under non-native operating systems
  • Understand binary obfuscation techniques
  • Identify and analyze anti-debugging and anti-analysis tricks

Who this book is for

If you are a security engineer or analyst or a system programmer and want to use reverse engineering to improve your software and hardware, this is the book for you. You will also find this book useful if you are a developer who wants to explore and learn reverse engineering. Having some programming/shell scripting knowledge is an added advantage.

Publisher Resources

Download Example Code

Table of Contents

  1. Title Page
  2. Copyright and Credits
    1. Mastering Reverse Engineering
  3. Packt Upsell
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Preparing to Reverse
    1. Reverse engineering
    2. Technical requirements
    3. Reverse engineering as a process
      1. Seeking approval
      2. Static analysis
      3. Dynamic analysis
      4. Low-level analysis
      5. Reporting
    4. Tools
      1. Binary analysis tools
      2. Disassemblers
      3. Debuggers
      4. Monitoring tools
      5. Decompilers
    5. Malware handling
    6. Basic analysis lab setup
      1. Our setup
    7. Samples
    8. Summary
  7. Identification and Extraction of Hidden Components
    1. Technical requirements
    2. The operating system environment
      1. The filesystem
      2. Memory
      3. The registry system
    3. Typical malware behavior
      1. Persistence
        1. Run keys
        2. Load and Run values
        3. Startup values
        4. The Image File Execution Options key
      2. Malware delivery
        1. Email
        2. Instant messenger
        3. The computer network
        4. Media storage
        5. Exploits and compromised websites
        6. Software piracy
          1. Malware file properties
      3. Payload – the evil within
    4. Tools
      1. Autoruns
      2. The Process explorer
    5. Summary
    6. Further reading
  8. The Low-Level Language
    1. Technical requirements
    2. Binary numbers
      1. Bases
      2. Converting between bases
      3. Binary arithmetic
      4. Signed numbers
    3. x86
      1. Registers
      2. Memory addressing
        1. Endianness
    4. Basic instructions
      1. Opcode bytes
      2. Copying data
        1. MOV and LEA
      3. Arithmetic operations
        1. Addition and subtraction
        2. Increment and decrement instructions
        3. Multiplication and division instructions
        4. Other signed operations
      4. Bitwise algebra
      5. Control flow
      6. Stack manipulation
    5. Tools – builder and debugger
      1. Popular assemblers
        1. MASM
        2. NASM
        3. FASM
      2. x86 Debuggers
        1. WinDbg
        2. Ollydebug
        3. x64dbg
    6. Hello World
      1. Installation of FASM
      2. It works!
      3. Dealing with common errors when building
      4. Dissecting the program 
    7. After Hello
      1. Calling APIs
        1. Common Windows API libraries
        2. Short list of common API functions
      2. Debugging
    8. Summary
    9. Further reading
  9. Static and Dynamic Reversing
    1. Assessment and static analysis
      1. Static analysis
      2. File types and header analysis
        1. Extracting useful information from file
          1. PEid and TrID
          2. python-magic
          3. file
          4. MASTIFF
      3. Other information
        1. PE executables
      4. Deadlisting
        1. IDA (Interactive Disassembler)
        2. Decompilers
          1. ILSpy – C# Decompiler
    2. Dynamic analysis
      1. Memory regions and the mapping of a process
      2. Process and thread monitoring
      3. Network traffic
      4. Monitoring system changes
      5. Post-execution differences
      6. Debugging
    3. Try it yourself
    4. Summary
    5. References
  10. Tools of the Trade
    1. Analysis environments
      1. Virtual machines
      2. Windows
      3. Linux
    2. Information gathering tools
      1. File type information
      2. Hash identifying
      3. Strings
      4. Monitoring tools
      5. Default command-line tools
    3. Disassemblers
    4. Debuggers
    5. Decompilers
    6. Network tools
    7. Editing tools
    8. Attack tools
    9. Automation tools
    10. Software forensic tools
    11. Automated dynamic analysis
    12. Online service sites
    13. Summary
  11. RE in Linux Platforms
    1. Setup
    2. Linux executable – hello world
      1. dlroW olleH
        1. What have we gathered so far?
        2. Dynamic analysis
        3. Going further with debugging
      2. A better debugger
        1. Setup
        2. Hello World in Radare2
      3. What is the password?
    3. Network traffic analysis
    4. Summary
    5. Further reading
  12. RE for Windows Platforms
    1. Technical requirements
    2. Hello World
      1. Learning about the APIs
        1. Keylogger
        2. regenum
        3. processlist
        4. Encrypting and decrypting a file
        5. The server
    3. What is the password?
      1. Static analysis
      2. A quick run
      3. Deadlisting
      4. Dynamic analysis with debugging
      5. Decompilers
    4. Summary
    5. Further reading
  13. Sandboxing - Virtualization as a Component for RE
    1. Emulation
      1. Emulation of Windows and Linux under an x86 host
        1. Emulators
    2. Analysis in unfamiliar environments
      1. Linux ARM guest in QEMU
      2. MBR debugging with Bochs
    3. Summary
    4. Further Reading
  14. Binary Obfuscation Techniques
    1. Data assembly on the stack
      1. Code assembly
    2. Encrypted data identification
      1. Loop codes
      2. Simple arithmetic
      3. Simple XOR decryption
    3. Assembly of data in other memory regions
    4. Decrypting with x86dbg
    5. Other obfuscation techniques
      1. Control flow flattening obfuscation
      2. Garbage code insertion
      3. Code obfuscation with a metamorphic engine
      4. Dynamic library loading
      5. Use of PEB information
    6. Summary
  15. Packing and Encryption
    1. A quick review on how native executables are loaded by the OS
    2. Packers, crypters, obfuscators, protectors and SFX
      1. Packers or compressors
      2. Crypters
      3. Obfuscators
      4. Protectors
      5. SFX  Self-extracting archives
    3. Unpacking
      1. The UPX tool
      2. Debugging though the packer
    4. Dumping processes from memory
      1. Memory dumping with VirtualBox
      2. Extracting the process to a file using Volatility
    5. How about an executable in its unpacked state?
    6. Other file-types
    7. Summary
  16. Anti-analysis Tricks
    1. Anti-debugging tricks
      1. IsDebuggerPresent
      2. Debug flags in the PEB
      3. Debugger information from NtQueryInformationProcess
      4. Timing tricks
      5. Passing code execution via SEH
        1. Causing exceptions
        2. A typical SEH setup
    2. Anti-VM tricks
      1. VM running process names
      2. Existence of VM files and directories
      3. Default MAC address
      4. Registry entries made by VMs
      5. VM devices
      6. CPUID results
    3. Anti-emulation tricks
    4. Anti-dumping tricks
    5. Summary
  17. Practical Reverse Engineering of a Windows Executable
    1. Things to prepare
    2. Initial static analysis
      1. Initial file information
      2. Deadlisting
    3. Debugging
      1. The unknown image
      2. Analysis summary
    4. Summary
    5. Further Reading
  18. Reversing Various File Types
    1. Analysis of HTML scripts
    2. MS Office macro analysis
    3. PDF file analysis
    4. SWF file analysis
      1. SWFTools
      2. FLASM
      3. Flare
      4. XXXSWF
      5. JPEXS SWF decompiler 
    5. Summary
    6. Further reading
  19. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product Information

  • Title: Mastering Reverse Engineering
  • Author(s): Reginald Wong
  • Release date: October 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788838849