book

Mastering the Lightning Network

by Andreas M. Antonopoulos, Olaoluwa Osuntokun, René Pickhardt

November 2021

Intermediate to advanced

464 pages

12h 6m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
Intended AudienceConventions Used in This BookCode ExamplesUsing Code ExamplesReferences to Companies and ProductsAddresses and Transactions in This BookO’Reilly Online LearningHow to Contact UsContacting AndreasContacting RenéContacting Olaoluwa OsuntokunAcknowledgments by AndreasAcknowledgments by RenéAcknowledgments by Olaoluwa OsuntokunContributionsSources
I. Understanding the Lightning Network
1. Introduction
Lightning Network Basic ConceptsTrust in Decentralized NetworksFairness Without Central AuthorityTrusted Protocols Without IntermediariesA Fairness Protocol in ActionSecurity Primitives as Building BlocksExample of the Fairness ProtocolMotivation for the Lightning NetworkScaling BlockchainsThe Lightning Network’s Defining FeaturesLightning Network Use Cases, Users, and Their StoriesConclusion
2. Getting Started
Alice’s First Lightning WalletLightning NodesLightning ExplorersLightning WalletsTestnet BitcoinBalancing Complexity and ControlDownloading and Installing a Lightning WalletCreating a New WalletResponsibility with Key CustodyMnemonic WordsStoring the Mnemonic SafelyLoading Bitcoin onto the WalletAcquiring BitcoinReceiving BitcoinFrom Bitcoin to Lightning NetworkLightning Network ChannelsOpening a Lightning ChannelBuying a Cup of Coffee Using the Lightning NetworkBob’s CafeA Lightning InvoiceConclusion
3. How the Lightning Network Works
What Is a Payment Channel?Payment Channel BasicsRouting Payments Across ChannelsPayment ChannelsMultisignature AddressFunding TransactionCommitment TransactionCheating with Prior StateAnnouncing the ChannelClosing the ChannelInvoicesPayment Hash and PreimageAdditional MetadataDelivering the PaymentThe Peer-to-Peer Gossip ProtocolPathfinding and RoutingSource-Based PathfindingOnion RoutingPayment Forwarding AlgorithmPeer-to-Peer Communication EncryptionThoughts About TrustComparison with BitcoinAddresses Versus Invoices, Transactions Versus PaymentsSelecting Outputs Versus Finding a PathChange Outputs on Bitcoin Versus No Change on LightningMining Fees Versus Routing FeesVarying Fees Depending on Traffic Versus Announced FeesPublic Bitcoin Transactions Versus Private Lightning PaymentsWaiting for Confirmations Versus Instant SettlementSending Arbitrary Amounts Versus Capacity RestrictionsIncentives for Large Value Payment Versus Small Value PaymentsUsing the Blockchain as a Ledger Versus as a Court SystemOffline Versus Online, Asynchronous Versus SynchronousSatoshis Versus MillisatoshisCommonality of Bitcoin and LightningMonetary UnitIrreversibility and Finality of PaymentsTrust and Counterparty RiskPermissionless OperationOpen Source and Open SystemConclusion
4. Lightning Node Software
Lightning Development EnvironmentUsing the Command LineDownloading the Book RepositoryDocker ContainersBitcoin Core and RegtestBuilding the Bitcoin Core ContainerThe c-lightning Lightning Node ProjectBuilding c-lightning as a Docker ContainerSetting Up a Docker NetworkRunning the bitcoind and c-lightning ContainersInstalling c-lightning from Source CodeInstalling Prerequisite Libraries and PackagesCopying the c-lightning Source CodeCompiling the c-lightning Source CodeThe Lightning Network Daemon Node ProjectThe LND Docker ContainerRunning the bitcoind and LND ContainersInstalling LND from Source CodeCopying the LND Source CodeCompiling the LND Source CodeThe Eclair Lightning Node ProjectThe Eclair Docker ContainerRunning the bitcoind and Eclair ContainersInstalling Eclair from Source CodeCopying the Eclair Source CodeCompiling the Eclair Source CodeBuilding a Complete Network of Diverse Lightning NodesUsing docker-compose to Orchestrate Docker Containersdocker-compose ConfigurationStarting the Example Lightning NetworkOpening Channels and Routing a PaymentConclusion
5. Operating a Lightning Network Node
Choosing Your PlatformWhy Is Reliability Important for Running a Lightning Node?Types of Hardware Lightning NodesRunning in the “Cloud”Running a Node at HomeWhat Hardware Is Required to Run a Lightning Node?Switching Server Configuration in the CloudUsing an Installer or HelperRaspiBlitzMynodeUmbrelBTCPay ServerBitcoin Node or Lightweight LightningOperating System ChoiceChoose Your Lightning Node ImplementationInstalling a Bitcoin or Lightning NodeBackground ServicesProcess IsolationNode StartupNode ConfigurationNetwork ConfigurationSecurity of Your NodeOperating System SecurityNode AccessNode and Channel BackupsHot Wallet RiskSweeping FundsLightning Node Uptime and AvailabilityTolerate Faults and AutomateMonitoring Node AvailabilityWatchtowersChannel ManagementOpening Outbound ChannelsGetting Inbound LiquidityClosing ChannelsRebalancing ChannelsRouting FeesNode ManagementRide The LightninglndmonThunderHubConclusion
II. The Lightning Network in Detail
6. Lightning Network Architecture
The Lightning Network Protocol SuiteLightning in Detail
7. Payment Channels
A Different Way of Using the Bitcoin SystemBitcoin Ownership and ControlDiversity of (Independent) Ownership and MultisigJoint Ownership Without Independent ControlPreventing “Locked” and Un-Spendable BitcoinConstructing a Payment ChannelNode Private and Public KeysNode Network AddressNode IdentifiersConnecting Nodes as Direct PeersConstructing the ChannelPeer Protocol for Channel ManagementChannel Establishment Message FlowThe Funding TransactionGenerating a Multisignature AddressConstructing the Funding TransactionHolding Signed Transactions Without BroadcastingRefund Before FundingConstructing the Presigned Refund TransactionChaining Transactions Without BroadcastingSolving Malleability (Segregated Witness)Broadcasting the Funding TransactionSending Payments Across the ChannelSplitting the BalanceCompeting CommitmentsCheating with Old Commitment TransactionsRevoking Old Commitment TransactionsAsymmetric Commitment TransactionsDelayed (Timelocked) Spending to_selfRevocation KeysThe Commitment TransactionAdvancing the Channel StateThe commitment_signed MessageThe revoke_and_ack MessageRevoking and RecommittingCheating and Penalty in PracticeThe Channel Reserve: Ensuring Skin in the GameClosing the Channel (Cooperative Close)The Shutdown MessageThe closing_signed MessageThe Cooperative Close TransactionConclusion

8. Routing on a Network of Payment Channels
Routing a PaymentRouting Versus PathfindingCreating a Network of Payment ChannelsA Physical Example of “Routing”Fairness ProtocolImplementing Atomic Trustless Multihop PaymentsRevisiting the Tipping ExampleOn-Chain Versus Off-Chain Settlement of HTLCsHash Time-Locked ContractsHTLCs in Bitcoin ScriptPayment Preimage and Hash VerificationExtending HTLCs from Alice to DinaBack-Propagating the SecretSignature Binding: Preventing Theft of HTLCsHash OptimizationHTLC Cooperative and Timeout FailureDecrementing TimelocksConclusion
9. Channel Operation and Payment Forwarding
Local (Single Channel) Versus Routed (Multiple Channels)Forwarding Payments and Updating Commitments with HTLCsHTLC and Commitment Message FlowForwarding Payments with HTLCsAdding an HTLCThe update_add_HTLC MessageHTLC in Commitment TransactionsNew Commitment with HTLC OutputAlice CommitsBob Acknowledges New Commitment and Revokes Old OneBob CommitsMultiple HTLCsHTLC FulfillmentHTLC PropagationDina Fulfills the HTLC with ChanBob Settles the HTLC with AliceRemoving an HTLC Due to Error or ExpiryMaking a Local PaymentConclusion
10. Onion Routing
A Physical Example Illustrating Onion RoutingSelecting a PathBuilding the LayersPeeling the LayersIntroduction to Onion Routing of HTLCsAlice Selects the PathAlice Constructs the PayloadsKey GenerationWrapping the Onion LayersFixed-Length OnionsWrapping the Onion (Outlined)Wrapping Dina’s Hop PayloadWrapping Chan’s Hop PayloadWrapping Bob’s Hop PayloadThe Final Onion PacketSending the OnionThe update_add_htlc MessageAlice Sends the Onion to BobBob Checks the OnionBob Generates FillerBob De-Obfuscates His Hop PayloadBob Extracts the Outer HMAC for the Next HopBob Removes His Payload and Left-Shifts the OnionBob Constructs the New Onion PacketBob Verifies the HTLC DetailsBob Sends the update_add_htlc to ChanChan Forwards the OnionDina Receives the Final PayloadReturning ErrorsFailure MessagesKeysend Spontaneous PaymentsCustom Onion TLV RecordsSending and Receiving Keysend PaymentsKeysend and Custom Records in Lightning ApplicationsConclusion
11. Gossip and the Channel Graph
Peer DiscoveryP2P BootstrappingDNS BootstrappingSRV Query OptionsThe Channel GraphA Directed GraphGossip Protocol MessagesThe node_announcement MessageThe channel_announcement MessageThe channel_update MessageOngoing Channel Graph MaintenanceConclusion
12. Pathfinding and Payment Delivery
Pathfinding in the Lightning Protocol SuiteWhere Is the BOLT?Pathfinding: What Problem Are We Solving?Selecting the Best PathPathfinding in Math and Computer ScienceCapacity, Balance, LiquidityUncertainty of BalancesPathfinding ComplexityKeeping It SimplePathfinding and Payment Delivery ProcessChannel Graph ConstructionLiquidity Uncertainty and ProbabilityFees and Other Channel MetricsFinding Candidate PathsPayment Delivery (Trial-and-Error Loop)First Attempt (Path #1)Second Attempt (Path #4)Multipart PaymentsUsing MPPTrial and Error over Multiple “Rounds”Conclusion
13. Wire Protocol: Framing and Extensibility
Messaging Layer in the Lightning Protocol SuiteWire FramingHigh-Level Wire FramingType EncodingType-Length-Value Message ExtensionsThe Protocol Buffers Message FormatForward and Backward CompatibilityType-Length-Value FormatBigSize Integer EncodingTLV Encoding ConstraintsTLV Canonical EncodingFeature Bits and Protocol ExtensibilityFeature Bits as an Upgrade Discoverability MechanismTLV for Forward and Backward CompatibilityA Taxonomy of Upgrade MechanismsChannel Construction-Level UpdatesConclusion
14. Lightning’s Encrypted Message Transport
Encrypted Transport in the Lightning Protocol SuiteIntroductionThe Channel Graph as Decentralized Public Key InfrastructureWhy Not TLS?The Noise Protocol FrameworkLightning Encrypted Transport in DetailNoise_XK: Lightning Network’s Noise HandshakeHandshake Notation and Protocol FlowHigh-Level OverviewHandshake in Three ActsConclusion
15. Lightning Payment Requests
Invoices in the Lightning Protocol SuiteIntroductionLightning Payment Requests Versus Bitcoin AddressesBOLT #11: Lightning Payment Request Serialization and InterpretationPayment Request Encoding in PracticeThe Human-Readable Prefixbech32 and the Data SegmentConclusion
16. Security and Privacy of the Lightning Network
Why Is Privacy Important?Definitions of PrivacyProcess to Evaluate PrivacyAnonymity SetDifferences Between the Lightning Network and Bitcoin in Terms of PrivacyAttacks on LightningObserving Payment AmountsLinking Senders and ReceiversRevealing Channel Balances (Probing)Denial of ServiceCommitment JammingChannel Liquidity LockupCross-Layer De-AnonymizationOn-Chain Bitcoin Entity ClusteringOff-Chain Lightning Node ClusteringCross-Layer Linking: Lightning Nodes and Bitcoin EntitiesLightning GraphHow Does the Lightning Graph Look in Reality?Centralization in the Lightning NetworkEconomic Incentives and Graph StructurePractical Advice for Users to Protect Their PrivacyUnannounced ChannelsRouting ConsiderationsAccepting ChannelsConclusionReferences and Further Reading
17. Conclusion
Decentralized and Asynchronous InnovationBitcoin Protocol and Bitcoin Script InnovationLightning Protocol InnovationTLV ExtensibilityPayment Channel ConstructionOpt-In End-to-End FeaturesLightning Applications (LApps)Ready, Set, Go!
A. Bitcoin Fundamentals Review
Keys and Digital SignaturesPrivate and Public KeysHashesDigital SignaturesSignature TypesBitcoin TransactionsInputs and OutputsTransaction ChainsTxID: Transaction IdentifiersOutpoints: Output IdentifiersBitcoin ScriptRunning Bitcoin ScriptLocking and Unlocking ScriptsLocking to a Public Key (Signature)Locking to a Hash (Secret)Multisignature ScriptsTimelock ScriptsScripts with Multiple ConditionsUsing Flow Control in Scripts
B. Docker Basic Installation and Use
Installing DockerBasic Docker CommandsBuilding a ContainerRunning a ContainerExecuting a Command in a ContainerStopping and Starting a ContainerDeleting a Container by NameListing Running ContainersListing Docker ImagesConclusion
C. Wire Protocol Messages
Message TypesMessage StructureConnection Establishment MessagesError Communication MessagesConnection LivenessChannel FundingChannel ClosingChannel OperationChannel AnnouncementChannel Graph Syncing
D. Sources and License Notices
SourcesBTCPay ServerLamassu Industries AG
Glossary
Index
About the Authors

Content preview from Mastering the Lightning Network

Appendix C. Wire Protocol Messages

This appendix lists all the currently defined message types used in the Lightning P2P protocol. Additionally, we show the structure of each message, grouping the messages into logical groupings based on the protocol flows.

Note

Lightning Protocol messages are extensible and their structure may change during network-wide upgrades. For the authoritative information, consult the latest version of the BOLTs found in the GitHub Lightning-RFC repository.

Message Types

Currently defined message types are listed in Table C-1.

Table C-1. Message types
Type integer	Message name	Category
16	`init`	Connection Establishment
17	`error`	Error Communication
18	`ping`	Connection Liveness
19	`pong`	Connection Liveness
32	`open_channel`	Channel Funding
33	`accept_channel`	Channel Funding
34	`funding_created`	Channel Funding
35	`funding_signed`	Channel Funding
36	`funding_locked`	Channel Funding + Channel Operation
38	`shutdown`	Channel Closing
39	`closing_signed`	Channel Closing
128	`update_add_htlc`	Channel Operation
130	`update_fulfill_hltc`	Channel Operation
131	`update_fail_htlc`	Channel Operation
132	`commit_sig`	Channel Operation
133	`revoke_and_ack`	Channel Operation
134	`update_fee`	Channel Operation
135	`update_fail_malformed_htlc`	Channel Operation
136	`channel_reestablish`	Channel Operation
256	`channel_announcement`	Channel Announcement
257	`node_announcement`	Channel Announcement
258	`channel_update`	Channel Announcement
259	`announce_signatures ...`

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781492054856Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design