Windows® Internals, Sixth Edition, Part 2

Windows® Internals, Sixth Edition, Part 2

by David A. Solomon Mark E. Russinovich and Alex Ionescu
Released September 2012
Publisher(s): Microsoft Press
ISBN: 9780735677265

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to live online training experiences, plus books, videos, and digital content from O’Reilly and nearly 200 trusted publishing partners.

Start your free trial

Book description

Delve inside Windows architecture and internals—and see how core components work behind the scenes. Led by three renowned internals experts, this classic guide is fully updated for Windows 7 and Windows Server 2008 R2—and now presents its coverage in two volumes.

As always, you get critical insider perspectives on how Windows operates. And through hands-on experiments, you’ll experience its internal behavior firsthand—knowledge you can apply to improve application design, debugging, system performance, and support.

In Part 2, you’ll examine:

  • Core subsystems for I/O, storage, memory management, cache manager, and file systems

  • Startup and shutdown processes

  • Crash-dump analysis, including troubleshooting tools and techniques

    •

    Table of contents

    1. Windows® Internals, Sixth Edition, Part 2
    2. Dedication
    3. Introduction
      1. Structure of the Book
      2. History of the Book
      3. Sixth Edition Changes
      4. Hands-on Experiments
      5. Topics Not Covered
      6. A Warning and a Caveat
      7. Acknowledgments
      8. Errata & Book Support
      9. We Want to Hear from You
      10. Stay in Touch
    4. 8. I/O System
      1. I/O System Components
        1. The I/O Manager
        2. Typical I/O Processing
      2. Device Drivers
        1. Types of Device Drivers
          1. WDM Drivers
          2. Layered Drivers
        2. Structure of a Driver
        3. Driver Objects and Device Objects
        4. Opening Devices
      3. I/O Processing
        1. Types of I/O
          1. Synchronous and Asynchronous I/O
          2. Fast I/O
          3. Mapped File I/O and File Caching
          4. Scatter/Gather I/O
          5. I/O Request Packets
          6. IRP Stack Locations
          7. IRP Buffer Management
        2. I/O Request to a Single-Layered Driver
          1. Servicing an Interrupt
          2. Completing an I/O Request
          3. Synchronization
        3. I/O Requests to Layered Drivers
          1. Thread Agnostic I/O
        4. I/O Cancellation
          1. User-Initiated I/O Cancellation
          2. I/O Cancellation for Thread Termination
        5. I/O Completion Ports
          1. The IoCompletion Object
          2. Using Completion Ports
          3. I/O Completion Port Operation
        6. I/O Prioritization
          1. I/O Priorities
          2. Prioritization Strategies
          3. I/O Priority Inversion Avoidance (I/O Priority Inheritance)
          4. I/O Priority Boosts and Bumps
          5. Bandwidth Reservation (Scheduled File I/O)
        7. Container Notifications
        8. Driver Verifier
      4. Kernel-Mode Driver Framework (KMDF)
        1. Structure and Operation of a KMDF Driver
        2. KMDF Data Model
        3. KMDF I/O Model
      5. User-Mode Driver Framework (UMDF)
      6. The Plug and Play (PnP) Manager
        1. Level of Plug and Play Support
        2. Driver Support for Plug and Play
        3. Driver Loading, Initialization, and Installation
          1. The Start Value
          2. Device Enumeration
          3. Device Stacks
          4. Device Stack Driver Loading
        4. Driver Installation
      7. The Power Manager
        1. Power Manager Operation
        2. Driver Power Operation
        3. Driver and Application Control of Device Power
        4. Power Availability Requests
        5. Processor Power Management (PPM)
          1. Core Parking Policies
          2. Utility Function
          3. Algorithm Overrides
          4. Increase/Decrease Actions
          5. Thresholds and Policy Settings
          6. Performance Check
      8. Conclusion
    5. 9. Storage Management
      1. Storage Terminology
      2. Disk Devices
        1. Rotating Magnetic Disks
          1. Disk Sector Format
        2. Solid State Disks
          1. NAND-Type Flash Memory
          2. File Deletion and the Trim Command
      3. Disk Drivers
        1. Winload
        2. Disk Class, Port, and Miniport Drivers
          1. iSCSI Drivers
          2. Multipath I/O (MPIO) Drivers
        3. Disk Device Objects
        4. Partition Manager
      4. Volume Management
        1. Basic Disks
          1. MBR-Style Partitioning
          2. GUID Partition Table Partitioning
          3. Basic Disk Volume Manager
        2. Dynamic Disks
          1. The LDM Database
          2. LDM and GPT or MBR-Style Partitioning
          3. Dynamic Disk Volume Manager
        3. Multipartition Volume Management
          1. Spanned Volumes
          2. Striped Volumes
          3. Mirrored Volumes
          4. RAID-5 Volumes
        4. The Volume Namespace
          1. The Mount Manager
          2. Mount Points
          3. Volume Mounting
        5. Volume I/O Operations
        6. Virtual Disk Service
      5. Virtual Hard Disk Support
        1. Attaching VHDs
        2. Nested File Systems
      6. BitLocker Drive Encryption
        1. Encryption Keys
        2. Trusted Platform Module (TPM)
        3. BitLocker Boot Process
        4. BitLocker Key Recovery
        5. Full-Volume Encryption Driver
        6. BitLocker Management
        7. BitLocker To Go
      7. Volume Shadow Copy Service
        1. Shadow Copies
          1. Clone Shadow Copies
          2. Copy-on-Write Shadow Copies
        2. VSS Architecture
        3. VSS Operation
          1. Shadow Copy Provider
        4. Uses in Windows
          1. Backup
          2. Previous Versions and System Restore
      8. Conclusion
    6. 10. Memory Management
      1. Introduction to the Memory Manager
        1. Memory Manager Components
        2. Internal Synchronization
        3. Examining Memory Usage
      2. Services Provided by the Memory Manager
        1. Large and Small Pages
        2. Reserving and Committing Pages
        3. Commit Limit
        4. Locking Memory
        5. Allocation Granularity
        6. Shared Memory and Mapped Files
        7. Protecting Memory
        8. No Execute Page Protection
          1. Software Data Execution Prevention
        9. Copy-on-Write
        10. Address Windowing Extensions
      3. Kernel-Mode Heaps (System Memory Pools)
        1. Pool Sizes
        2. Monitoring Pool Usage
        3. Look-Aside Lists
      4. Heap Manager
        1. Types of Heaps
        2. Heap Manager Structure
        3. Heap Synchronization
        4. The Low Fragmentation Heap
        5. Heap Security Features
        6. Heap Debugging Features
        7. Pageheap
        8. Fault Tolerant Heap
      5. Virtual Address Space Layouts
        1. x86 Address Space Layouts
        2. x86 System Address Space Layout
        3. x86 Session Space
        4. System Page Table Entries
        5. 64-Bit Address Space Layouts
        6. x64 Virtual Addressing Limitations
          1. Windows x64 16-TB Limitation
        7. Dynamic System Virtual Address Space Management
        8. System Virtual Address Space Quotas
        9. User Address Space Layout
          1. Image Randomization
          2. Stack Randomization
          3. Heap Randomization
          4. ASLR in Kernel Address Space
          5. Controlling Security Mitigations
      6. Address Translation
        1. x86 Virtual Address Translation
          1. Page Directories
          2. Page Tables and Page Table Entries
          3. Hardware vs. Software Write Bits in Page Table Entries
          4. Byte Within Page
        2. Translation Look-Aside Buffer
        3. Physical Address Extension (PAE)
        4. x64 Virtual Address Translation
        5. IA64 Virtual Address Translation
      7. Page Fault Handling
        1. Invalid PTEs
        2. Prototype PTEs
        3. In-Paging I/O
        4. Collided Page Faults
        5. Clustered Page Faults
        6. Page Files
        7. Commit Charge and the System Commit Limit
        8. Commit Charge and Page File Size
      8. Stacks
        1. User Stacks
        2. Kernel Stacks
        3. DPC Stack
      9. Virtual Address Descriptors
        1. Process VADs
        2. Rotate VADs
      10. NUMA
      11. Section Objects
      12. Driver Verifier
      13. Page Frame Number Database
        1. Page List Dynamics
        2. Page Priority
        3. Modified Page Writer
        4. PFN Data Structures
      14. Physical Memory Limits
        1. Windows Client Memory Limits
          1. 32-Bit Client Effective Memory Limits
      15. Working Sets
        1. Demand Paging
        2. Logical Prefetcher
        3. Placement Policy
        4. Working Set Management
        5. Balance Set Manager and Swapper
        6. System Working Sets
        7. Memory Notification Events
      16. Proactive Memory Management (Superfetch)
        1. Components
        2. Tracing and Logging
        3. Scenarios
        4. Page Priority and Rebalancing
        5. Robust Performance
        6. ReadyBoost
        7. ReadyDrive
        8. Unified Caching
        9. Process Reflection
      17. Conclusion
    7. 11. Cache Manager
      1. Key Features of the Cache Manager
        1. Single, Centralized System Cache
        2. The Memory Manager
        3. Cache Coherency
        4. Virtual Block Caching
        5. Stream-Based Caching
        6. Recoverable File System Support
      2. Cache Virtual Memory Management
      3. Cache Size
        1. Cache Virtual Size
        2. Cache Working Set Size
        3. Cache Physical Size
      4. Cache Data Structures
        1. Systemwide Cache Data Structures
        2. Per-File Cache Data Structures
      5. File System Interfaces
        1. Copying to and from the Cache
        2. Caching with the Mapping and Pinning Interfaces
        3. Caching with the Direct Memory Access Interfaces
      6. Fast I/O
      7. Read-Ahead and Write-Behind
        1. Intelligent Read-Ahead
        2. Write-Back Caching and Lazy Writing
          1. Disabling Lazy Writing for a File
          2. Forcing the Cache to Write Through to Disk
          3. Flushing Mapped Files
        3. Write Throttling
        4. System Threads
      8. Conclusion
    8. 12. File Systems
      1. Windows File System Formats
        1. CDFS
        2. UDF
        3. FAT12, FAT16, and FAT32
        4. exFAT
        5. NTFS
      2. File System Driver Architecture
        1. Local FSDs
        2. Remote FSDs
          1. Locking
        3. File System Operation
          1. Explicit File I/O
          2. Memory Manager’s Modified and Mapped Page Writer
          3. Cache Manager’s Lazy Writer
          4. Cache Manager’s Read-Ahead Thread
          5. Memory Manager’s Page Fault Handler
        4. File System Filter Drivers
          1. Process Monitor
      3. Troubleshooting File System Problems
        1. Process Monitor Basic vs. Advanced Modes
        2. Process Monitor Troubleshooting Techniques
      4. Common Log File System
        1. Marshalling
          1. Marshalling
          2. Log Types
          3. Log Layout
          4. Log Sequence Numbers
          5. Log Blocks
          6. Owner Pages
          7. Translating Virtual LSNs to Physical LSNs
          8. Management Policies
      5. NTFS Design Goals and Features
        1. High-End File System Requirements
          1. Recoverability
          2. Security
          3. Data Redundancy and Fault Tolerance
        2. Advanced Features of NTFS
          1. Multiple Data Streams
          2. Unicode-Based Names
          3. General Indexing Facility
          4. Dynamic Bad-Cluster Remapping
          5. Hard Links
          6. Symbolic (Soft) Links and Junctions
          7. Compression and Sparse Files
          8. Change Logging
          9. Per-User Volume Quotas
          10. Link Tracking
          11. Encryption
          12. POSIX Support
          13. Defragmentation
          14. Dynamic Partitioning
      6. NTFS File System Driver
      7. NTFS On-Disk Structure
        1. Volumes
        2. Clusters
        3. Master File Table
        4. File Record Numbers
        5. File Records
        6. File Names
        7. Resident and Nonresident Attributes
        8. Data Compression and Sparse Files
          1. Compressing Sparse Data
          2. Compressing Nonsparse Data
          3. Sparse Files
        9. The Change Journal File
        10. Indexing
        11. Object IDs
        12. Quota Tracking
        13. Consolidated Security
        14. Reparse Points
        15. Transaction Support
          1. Isolation
          2. Transactional APIs
          3. Resource Managers
          4. On-Disk Implementation
          5. Logging Implementation
          6. Recovery Implementation
      8. NTFS Recovery Support
        1. Design
        2. Metadata Logging
          1. Log File Service
          2. Log Record Types
        3. Recovery
          1. Analysis Pass
          2. Redo Pass
          3. Undo Pass
        4. NTFS Bad-Cluster Recovery
        5. Self-Healing
      9. Encrypting File System Security
        1. Encrypting a File for the First Time
          1. Encrypting File Data
        2. The Decryption Process
        3. Backing Up Encrypted Files
        4. Copying Encrypted Files
      10. Conclusion
    9. 13. Startup and Shutdown
      1. Boot Process
        1. BIOS Preboot
        2. The BIOS Boot Sector and Bootmgr
        3. The UEFI Boot Process
        4. Booting from iSCSI
        5. Initializing the Kernel and Executive Subsystems
        6. Smss, Csrss, and Wininit
        7. ReadyBoot
        8. Images That Start Automatically
      2. Troubleshooting Boot and Startup Problems
        1. Last Known Good
        2. Safe Mode
          1. Driver Loading in Safe Mode
          2. Safe-Mode-Aware User Programs
          3. Boot Logging in Safe Mode
        3. Windows Recovery Environment (WinRE)
        4. Solving Common Boot Problems
          1. MBR Corruption
          2. Boot Sector Corruption
          3. BCD Misconfiguration
          4. System File Corruption
          5. System Hive Corruption
          6. Post–Splash Screen Crash or Hang
      3. Shutdown
      4. Conclusion
    10. 14. Crash Dump Analysis
      1. Why Does Windows Crash?
      2. The Blue Screen
        1. Causes of Windows Crashes
      3. Troubleshooting Crashes
      4. Crash Dump Files
        1. Crash Dump Generation
      5. Windows Error Reporting
      6. Online Crash Analysis
      7. Basic Crash Dump Analysis
        1. Notmyfault
        2. Basic Crash Dump Analysis
        3. Verbose Analysis
      8. Using Crash Troubleshooting Tools
        1. Buffer Overruns, Memory Corruption, and Special Pool
        2. Code Overwrite and System Code Write Protection
      9. Advanced Crash Dump Analysis
        1. Stack Trashes
        2. Hung or Unresponsive Systems
        3. When There Is No Crash Dump
      10. Analysis of Common Stop Codes
        1. 0xD1 - DRIVER_IRQL_NOT_LESS_OR_EQUAL
        2. 0x8E - KERNEL_MODE_EXCEPTION_NOT_HANDLED
        3. 0x7F - UNEXPECTED_KERNEL_MODE_TRAP
        4. 0xC5 - DRIVER_CORRUPTED_EXPOOL
        5. Hardware Malfunctions
      11. Conclusion
    11. A. Contents of Windows Internals, Sixth Edition, Part 1
    12. Index
    13. About the Authors
    14. Copyright

    Product information

    • Title: Windows® Internals, Sixth Edition, Part 2
    • Author(s): David A. Solomon Mark E. Russinovich and Alex Ionescu
    • Release date: September 2012
    • Publisher(s): Microsoft Press
    • ISBN: 9780735677265