Mastering the Lightning Network

Book description

Launched in early 2018, the Lightning Network (LN) is rapidly growing in users and capacity. This second-layer payment protocol works on top of Bitcoin and other cryptocurrencies to provide near-instantaneous transactions between two parties. With this practical guide, authors Andreas M. Antonopoulos, Olaoluwa Osuntokun, and Rene Pickhardt explain how this advancement will enable the next level of scale for Bitcoin, increasing speed and privacy while reducing fees.

Ideal for developers, systems architects, investors, and entrepreneurs looking to gain a better understanding of LN, this book demonstrates why experts consider LN a critical solution to Bitcoin's scalability problem. You'll learn how LN has the potential to support far more transactions than today's financial networks, ushering in an era of global micro-transactions at sub-second resolution.

In several parts, this book examines:

  • The challenges of scaling blockchain technology and why the Lightning Network was invented
  • LN basics including wallets, nodes, and lightning payments
  • Lightning payment channels and how they work
  • Routing payments by constructing paths of payment channels from sender to recipient, including onion routing, and atomic multi-path payments
  • Lightning developments such as eltoo, Schnorr signatures, HODL invoices, JIT routing, channel splicing, and channel factories
  • Building applications on Lightning (Lapps)

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. How to Use This Book
    2. Intended Audience
    3. Conventions Used in This Book
    4. Code Examples
    5. Using Code Examples
    6. References to Companies and Products
    7. Addresses and Transactions in this Book
    8. O’Reilly Online Learning
    9. How to Contact Us
      1. Contacting Andreas
      2. Contacting René
    10. Acknowledgments by Andreas
    11. Acknowledgments by René
    12. Contributions
    13. Sources
  2. I. Understanding The Lightning Network
  3. 1. Introduction
    1. Lightning Network Basic Concepts
    2. Trust in decentralized networks
    3. Fairness without central authority
      1. Trusted protocols without intermediaries
      2. A fairness protocol in action
      3. Security primitives as building blocks
      4. Example of the fairness protocol
    4. Motivation for the Lightning Network
      1. Scaling Blockchains
    5. The Lightning Network’s defining features
    6. Lightning Network Use Cases, Users, and Their Stories
    7. Chapter Summary
  4. 2. Getting started
    1. Alice’s first Lightning wallet
    2. Lightning nodes
    3. Lightning explorers
    4. Lightning wallets
      1. Testnet Bitcoin
    5. Balancing complexity and control
    6. Downloading and installing a Lightning wallet
    7. Creating a new wallet
      1. Responsibility with key custody
      2. Mnemonic words
      3. Storing the mnemonic safely
    8. Loading bitcoin onto the wallet
      1. Acquiring bitcoin
      2. Receiving bitcoin
    9. From Bitcoin to Lightning network
      1. Lightning Network channels
      2. Opening a Lightning channel
    10. Buying a cup of coffee using LN
      1. Bob’s Cafe
      2. A Lightning invoice
    11. Conclusion
  5. 3. How the Lightning Network works
    1. What is a payment channel?
    2. Payment channel basics
    3. Routing payments across channels
    4. Payment channels
      1. Multisignature address
      2. Funding transaction
      3. Commitment Transaction
      4. Cheating with prior state
      5. Announcing the channel
      6. Closing the channel
    5. Invoices
      1. Payment hash and preimage
      2. Additional metadata
    6. Delivering the payment
      1. The peer-to-peer gossip protocol
      2. Pathfinding and routing
    7. Source-based pathfinding
      1. Onion routing
      2. Payment forwarding algorithm
    8. Peer-to-peer communication encryption
    9. Thoughts about trust
    10. Comparison with Bitcoin
      1. Addresses vs. invoices, transactions vs. payments
      2. Selecting outputs vs. finding a path
      3. Change outputs on Bitcoin vs. no change on Lightning
      4. Mining fees vs. routing fees
      5. Varying fees depending on traffic vs. announced fees
      6. Public Bitcoin transactions vs. private Lightning payments
      7. Waiting for confirmations vs. instant settlement
      8. Sending arbitrary amounts vs. capacity restrictions
      9. Incentives for large value payment vs. small value payments
      10. Using the Blockchain as a ledger vs. as a court system
      11. Offline vs. online, asynchronous vs. synchronous
      12. Satoshis vs. milli-satoshis
    11. Commonality of Bitcoin and Lightning
      1. Monetary unit
      2. Irreversibility and finality of payments
      3. Trust and counterparty risk
      4. Permissionless operation
      5. Open source and open system
    12. Conclusion
  6. 4. Lightning node software
    1. Lightning development environment
      1. Using the command-line
      2. Downloading the book repository
    2. Docker containers
    3. Bitcoin Core and regtest
      1. Building the Bitcoin Core container
    4. The c-lightning Lightning node project
      1. Building c-lightning as a Docker container
      2. Setting up a Docker network
      3. Running the bitcoind and c-lightning containers
      4. Installing c-lightning from source code
      5. Installing prerequisite libraries and packages
      6. Copying the c-lightning source code
      7. Compiling the c-lightning source code
    5. The Lightning Network Daemon (LND) node project
      1. The LND docker container
      2. Running the bitcoind and LND containers
      3. Installing LND from source code
      4. Copying the LND source code
      5. Compiling the LND source code
    6. The Eclair Lightning node project
      1. The Eclair docker container
      2. Running the bitcoind and Eclair containers
      3. Installing Eclair from source code
      4. Copying the Eclair source code
      5. Compiling the Eclair source code
    7. Building a complete network of diverse Lightning nodes
      1. Using docker-compose to orchestrate Docker containers
      2. Docker-compose configuration
      3. Starting the example Lightning network
      4. Opening channels and routing a payment
    8. Conclusion
  7. 5. Operating a Lightning Network node
    1. Choosing your platform
      1. Why is reliability important for running a Lightning node?
      2. Types of hardware Lightning nodes
      3. Running in the “cloud”
      4. Running a node at home
      5. What hardware is required to run a Lightning node?
      6. Switching server configuration in the cloud
    2. Using an installer or helper
      1. RaspiBlitz
      2. myNode
      3. Umbrel
      4. BTCPay Server
      5. Bitcoin node or lightweight Lightning
      6. Operating system choice
    3. Choose your Lightning node implementation
    4. Installing a Bitcoin or Lightning node
      1. Background services
      2. Process isolation
      3. Node startup
      4. Node configuration
      5. Network configuration
    5. Security of your node
      1. Operating system security
      2. Node access
    6. Node and channel backups
      1. Static Channel Backups (SCB)
      2. Hot wallet risk
      3. Sweeping funds
    7. Lightning node uptime and availability
      1. Tolerate faults and automate
      2. Monitoring node availability
      3. Watchtowers
    8. Channel management
      1. Opening outbound channels
      2. Getting inbound liquidity
      3. Closing channels
      4. Re-balancing channels
    9. Routing fees
    10. Node management
      1. Ride The Lightning (RTL)
      2. LNDMon
      3. ThunderHub
    11. Conclusion
  8. II. The Lightning Network in detail
  9. 6. Lightning Network Architecture
    1. The Lightning Network Protocol Suite
    2. Lightning in Detail
  10. 7. Payment Channels
    1. A different way of using the Bitcoin system
    2. Bitcoin ownership and control
      1. Diversity of (independent) ownership and multisig
      2. Joint ownership without independent control
      3. Preventing “locked” and un-spendable Bitcoin
    3. Constructing a payment channel
      1. Node private and public keys
      2. Node network address
      3. Node identifiers
      4. Connecting nodes as direct peers
    4. Constructing the channel
      1. Peer Protocol for Channel Management
      2. Channel establishment message flow
      3. The funding transaction
      4. Generating a multisignature address
      5. Constructing the funding transaction
      6. Holding signed transactions without broadcasting
      7. Refund before funding
      8. Constructing the pre-signed refund transaction
      9. Chaining transactions without broadcasting
      10. Solving malleability (Segregated Witness)
      11. Broadcasting the funding transaction
    5. Sending payments across the channel
      1. Splitting the balance
      2. Competing commitments
      3. Cheating with old commitment transactions
      4. Revoking old commitment transactions
      5. Asymmetric commitment transactions
      6. Delayed (timelocked) spending to_self
      7. Revocation keys
    6. The commitment transaction
    7. Advancing the channel state
      1. The commitment_signed message
      2. The revoke_and_ack message
      3. Revoking and re-committing
      4. Cheating and penalty in practice
      5. The channel reserve: ensuring skin in the game
    8. Closing the channel (cooperative close)
      1. The shutdown message
      2. The closing_signed message
      3. The cooperative close transaction
    9. Conclusion
  11. 8. Routing on a Network of Payment channels
    1. Routing a payment
    2. Routing vs. Path Finding
    3. Creating a network of payment channels
      1. A physical example of “routing”
    4. Fairness Protocol
      1. Implementing Atomic Trustless Multihop Payments
    5. Revisiting our example
      1. On-chain vs Off-chain settlement of HTLCs
    6. Hash Time Locked Contracts (HTLCs)
      1. HTLCs in Bitcoin Script
      2. Payment pre-image and hash verification
      3. Extending HTLCs from Alice to Dina
      4. Back-propagating the secret
      5. Signature binding - preventing theft of HTLCs
      6. Hash Optimization
      7. HTLC cooperative and timeout failure
      8. Decrementing time-locks
    7. Conclusion
  12. 9. Channel Operation and Payment Forwarding
    1. Local (single channel) vs Routed (multiple channels)
    2. Forwarding payments and updating commitments with HTLCs
      1. HTLC and commitment message flow
    3. Forwarding payments with HTLCs
      1. Adding an HTLC
      2. The update_add_htlc message
      3. HTLC in commitment transaction
      4. New commitment with HTLC output
      5. Alice commits
      6. Bob acknowledges new commitment and revokes old one
      7. Bob commits
    4. Multiple HTLCs
    5. HTLC fulfillment
      1. HTLC propagation
      2. Dina fulfills the HTLC with Chan
      3. Bob settles the HTLC with Alice
    6. Removing an HTLC due to error or expiry
    7. Making a local payment
    8. Conclusion
  13. 10. Onion Routing
    1. A physical example illustrating onion routing
      1. Selecting a path
      2. Building the layers
      3. Peeling the layers
    2. Introduction to onion routing of HTLCs
      1. Alice selects the path
      2. Alice constructs the payloads
      3. Key generation
    3. Wrapping the onion layers
      1. Fixed-length Onions
      2. Wrapping the onion (outlined)
      3. Wrapping Dina’s hop payload
      4. Wrapping Chan’s hop payload
      5. Wrapping Bob’s hop payload
      6. The final onion packet
    4. Sending the onion
      1. The update_add_htlc message
      2. Alice sends the onion to Bob
      3. Bob checks the onion
      4. Bob generates filler
      5. Bob de-obfuscates his hop payload
      6. Bob extracts the outer HMAC for the next hop
      7. Bob removes his payload and left shifts the onion
      8. Bob constructs the new onion packet
      9. Bob verifies the HTLC details
      10. Bob sends the update_add_htlc to Chan
      11. Chan forwards the onion
      12. Dina receives the final payload
    5. Returning errors
      1. Failure Messages
    6. Conclusion
  14. 11. Gossip and the Channel Graph
    1. Peer Discovery
      1. P2P Bootstrapping
      2. DNS Bootstrapping
      3. SRV Query Options
    2. The channel graph
      1. A directed graph
    3. Gossip protocol messages
      1. The node_announcement message
      2. The channel_announcement message
      3. The channel_update message
    4. Ongoing Channel Graph Maintenance
    5. Conclusion
  15. 12. Path Finding and Payment Delivery
    1. Path finding in the Lightning protocol suite
      1. Where is the BOLT?
    2. Path finding: what problem are we solving?
      1. Selecting the best path
      2. Path finding in math and computer science
      3. Capacity, balance, liquidity
      4. Uncertainty of balances
      5. Path finding complexity
      6. Keeping it simple
    3. Path finding and payment delivery process
    4. Channel graph construction
      1. Liquidity uncertainty and probability
      2. Fees and other channel metrics
    5. Finding candidate paths
    6. Payment delivery (Trial-and-error loop)
      1. First attempt (path #1)
      2. Second attempt (path #4)
    7. Multi-Path Payments (MPP)
      1. Using MPP
      2. Trial-and-error over multiple “rounds”
    8. Conclusion
  16. 13. Wire Protocol: Framing & Extensibility
    1. Messaging Layer in the Lightning protocol suite
    2. Wire Framing
      1. High-Level Wire Framing
      2. Type Encoding
    3. Type Length Value (TLV) Message Extensions
      1. The Protocol Buffer Message Format
      2. Forwards & Backwards Compatibility
    4. Type-Length-Value (TLV) Format
      1. BigSize Integer Encoding
      2. TLV encoding constraints
      3. TLV canonical encoding
    5. Feature Bits & Protocol Extensibility
      1. Feature Bits as an Upgrade Discoverability Mechanism
      2. TLV for forwards & backwards compatibility
      3. A taxonomy of upgrade mechanisms
      4. Channel Construction Level Updates
    6. Conclusion
  17. 14. Lightning’s Encrypted Message Transport (Brontide)
    1. Encrypted Transport in the Lightning Protocol Suite
    2. Introduction
    3. The Channel Graph as Decentralized Public Key Infrastructure
    4. Why Not TLS?
    5. The Noise Protocol Framework
    6. Lightning Encrypted Transport in detail
      1. Noise XK: LN’s Noise Handshake
      2. Handshake Notation & Protocol Flow
      3. Handshake in Three Acts
    7. Conclusion
  18. 15. Lightning Payment Requests
    1. Invoices in the Lightning Protocol Suite
    2. Introduction
    3. Lightning Payment Requests vs Bitcoin Addresses
    4. BOLT 11: Lightning Payment Request Serialization & Interpretation
      1. Payment Request Encoding in Practice
      2. The Human Readable Prefix
      3. Bech32 & the Data Segment
    5. Conclusion
  19. 16. Security and Privacy of the Lightning Network
    1. Why is privacy important?
    2. Definitions of privacy
      1. Process to evaluate privacy
    3. Anonymity set
    4. Differences between LN and Bitcoin in terms of privacy
    5. Attacks on Lightning
      1. Observing payment amounts
      2. Linking senders and receivers
      3. Revealing channel balances (probing)
      4. Denial of service
      5. Commitment jamming
      6. Channel liquidity lockup
    6. Cross-layer deanonymization
      1. On-Chain Bitcoin Entity Clustering
      2. Off-Chain Lightning Node Clustering
      3. Cross-Layer Linking: Lightning Nodes and Bitcoin Entities
    7. Lightning graph
      1. How does the Lightning graph look in reality?
    8. Centralization in the LN
    9. Economic incentives and graph structure
    10. Practical advice for users to protect their privacy
    11. Unannounced channels
    12. Routing considerations
      1. Accepting channels
    13. Conclusion
    14. References and further reading
  20. 17. Conclusion
    1. Decentralized and asynchronous innovation
      1. Bitcoin protocol and Bitcoin Script innovation
      2. Lightning protocol innovation
      3. Payment channel construction
      4. Opt-in end-to-end features
    2. Lightning Applications (LApps)
    3. Ready, set, go!
  21. A. Bitcoin Fundamentals Review
    1. Keys and digital signatures
      1. Private and public keys
      2. Hashes
      3. Digital signatures
      4. Signature Types
    2. Bitcoin transactions
      1. Inputs and outputs
      2. Transaction chains
      3. TxID: Transaction identifiers
      4. Outpoints: output identifiers
    3. Bitcoin Script
      1. Running Bitcoin Script
      2. Locking and Unlocking Scripts
      3. Locking to a public key (signature)
      4. Locking to a hash (secret)
      5. Multisignature scripts
      6. Timelock scripts
      7. Scripts with multiple conditions
      8. Using Flow Control in Scripts
  22. B. Docker Basic Installation and Use
    1. Installing Docker
    2. Basic Docker commands
    3. Conclusion
  23. C. Wire Protocol Messages
    1. Message Types
    2. Message Structure
      1. Connection Establishment Messages
      2. Error Communication Messages
      3. Connection Liveness
      4. Channel Funding
      5. Channel Closing
      6. Channel Operation
      7. Channel Announcement
      8. Channel Graph Syncing
  24. D. License Notices
    1. BTCPay Server
    2. Lamassu Industries AG
  25. Glossary

Product information

  • Title: Mastering the Lightning Network
  • Author(s): Andreas M. Antonopoulos, Olaoluwa Osuntokun, René Pickhardt
  • Release date: November 2021
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492054863