O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

x64 Windows Debugging: Practical Foundations

Book Description

Written by the founWritten by the founder of DumpAnalysis.org this book is not about bugs or debugging techniques but about background knowledge everyone needs to start experimenting with x64 WinDbg, learn from practical experience and read other advanced debugging books. Solid understanding of fundamentals like pointers is needed to analyze stack traces beyond !analyze -v and lmv WinDbg commands. This is the book to help technical support and escalation engineers and Windows software testers without the knowledge of assembly language to master necessary prerequisites to understand and start debugging and crash dump analysis on x64 Windows platforms. It doesn't require any specific knowledge, fills the gap and lowers the learning curve. The book is also useful for software engineers coming from managed code or Java background, engineers coming from non-Wintel environments, Windows C/C++ software engineers without assembly language background, security researchers and beginners learning x64 Windows software disassembling and reverse engineering techniques. This book can also be used as AMD64 and Intel EM64T assembly language and x64 Windows debugging supplement for relevant undergraduate level courses. For someone, who wants to learn these foundations in the context of 32-bit Windows environments there is a separate x86 book (ISBN: 978-1-906717-10-0). However, this book is completely independent from that earlier book and almost every illustration was recreated to reflect x64 architecture and x64 Windows ILP 32-32-64 model (Integer-Long-Pointer).

Table of Contents

  1. Copyright
  2. Preface
  3. Acknowledgements
  4. About the Author
  5. 1. Memory, Registers and Simple Arithmetic
    1. Memory and Registers inside an Idealized Computer
    2. Memory and Registers inside Intel 64-bit PC
    3. "Arithmetic" Project: Memory Layout and Registers
    4. "Arithmetic" Project: A Computer Program
    5. "Arithmetic" Project: Assigning Numbers to Memory Locations
    6. Assigning Numbers to Registers
    7. "Arithmetic" Project: Adding Numbers to Memory Cells
    8. Incrementing/Decrementing Numbers in Memory and Registers
    9. Multiplying Numbers
  6. 2. Debug and Release Binaries
    1. "Arithmetic" Project: C/C++ Program
    2. Downloading and Configuring WinDbg Debugger
    3. WinDbg Disassembly Output – Debug Executable
    4. WinDbg Disassembly Output – Release Executable
  7. 3. Number Representations
    1. Numbers and Their Representations
    2. Decimal Representation (Base Ten)
    3. Ternary Representation (Base Three)
    4. Binary Representation (Base Two)
    5. Hexadecimal Representation (Base Sixteen)
    6. Why Hexadecimals are used?
  8. 4. Pointers
    1. A Definition
    2. "Pointers" Project: Memory Layout and Registers
    3. "Pointers" Project: Calculations
    4. Using Pointers to Assign Numbers to Memory Cells
    5. Adding Numbers Using Pointers
    6. Multiplying Numbers Using Pointers
  9. 5. Bytes, Words and Double Words
    1. Using Hexadecimal Numbers
    2. Byte Granularity
    3. Bit Granularity
    4. Memory Layout
  10. 6. Pointers to Memory
    1. Pointers Revisited
    2. Addressing Types
    3. Registers Revisited
    4. NULL Pointers
    5. Invalid Pointers
    6. Variables as Pointers
    7. Pointer Initialization
    8. Note: Initialized and Uninitialized Data
    9. More Pseudo Notation
    10. "MemoryPointers" Project: Memory Layout
  11. 7. Logical Instructions and EIP
    1. Instruction Format
    2. Logical Shift Instructions
    3. Logical Operations
    4. Zeroing Memory or Registers
    5. Instruction Pointer
    6. Note: Code Section
  12. 8. Reconstructing a Program with Pointers
    1. Example of Disassembly Output: No Optimization
    2. Reconstructing C/C++ Code: Part 1
    3. Reconstructing C/C++ Code: Part 2
    4. Reconstructing C/C++ Code: Part 3
    5. Reconstructing C/C++ Code: C/C++ program
    6. Example of Disassembly Output: Optimized Program
  13. 9. Memory and Stacks
    1. Stack: A Definition
    2. Stack Implementation in Memory
    3. Things to Remember
    4. PUSH Instruction
    5. POP instruction
    6. Register Review
    7. Application Memory Simplified
    8. Stack Overflow
    9. Jumps
    10. Calls
    11. Call Stack
    12. Exploring Stack in WinDbg
  14. 10. Local Variables
    1. Stack Usage
    2. Addressing Array Elements
    3. Stack Structure (No Function Parameters)
    4. Function Prolog
    5. Function Epilog
    6. "Local Variables" Project
    7. Disassembly of Optimized Executable (Release Configuration)
  15. 11. Function Parameters
    1. "FunctionParameters" Project
    2. Stack Structure
    3. Function Prolog and Epilog
    4. Project Disassembled Code with Comments
    5. Parameter Mismatch Problem
  16. 12. More Instructions
    1. CPU Flags Register
    2. The Fastest Way to Fill Memory
    3. Testing for 0
    4. TEST - Logical Compare
    5. CMP – Compare Two Operands
    6. TEST or CMP?
    7. Conditional Jumps
    8. The Structure of Registers
    9. Function Return Value
    10. Using Byte Registers
  17. 13. Function Pointer Parameters
    1. "FunctionPointerParameters" Project
    2. Commented Disassembly
  18. 14. Summary of Code Disassembly Patterns
    1. Function Prolog / Epilog
    2. Parameters and Local Variables
    3. LEA (Load Effective Address)
    4. Accessing Parameters and Local Variables