book

Mastering Blockchain - Third Edition

by Imran Bashir

August 2020

Beginner

816 pages

23h 28m

English

Packt Publishing

Read now

Unlock full access

Who this book is forWhat this book coversTo get the most out of this bookGet in touch
The growth of blockchain technologyProgress toward maturityIncreasing interestDistributed systemsThe history of blockchain and BitcoinThe events that led to blockchainElectronic cashBlockchainBlockchain definedPeer-to-peerDistributed ledgerCryptographically secureAppend-onlyUpdatable via consensusBlockchain architectureBlockchain by layersBlockchain in businessGeneric elements of a blockchainHow blockchain worksBenefits, features, and limitations of blockchainTypes of blockchainDistributed ledgersDistributed Ledger TechnologyPublic blockchainsPrivate blockchainsSemi-private blockchainsSidechainsPermissioned ledgerShared ledgerFully private and proprietary blockchainsTokenized blockchainsTokenless blockchainsConsensusConsensus mechanismTypes of consensus mechanismsConsensus in blockchainCAP theorem and blockchainSummary
Decentralization using blockchainMethods of decentralizationDisintermediationContest-driven decentralization Routes to decentralizationHow to decentralizeDecentralization framework exampleBlockchain and full ecosystem decentralizationStorageCommunicationComputing power and decentralizationPertinent terminologySmart contractsAutonomous agentsDecentralized organizationsDecentralized autonomous organizationsDecentralized autonomous corporationsDecentralized autonomous societiesDecentralized applicationsRequirements of a DAppOperations of a DAppDesign of a DAppDApp examplesPlatforms for decentralizationEthereumMaidSafeLiskEOSInnovative trendsDecentralized webWeb 1Web 2Web 3Decentralized identityDecentralized finance (DeFi)Summary
Working with the OpenSSL command lineIntroductionCryptographyConfidentialityIntegrityAuthenticationNon-repudiationAccountabilityCryptographic primitivesKeyless primitivesRandom numbersHash functionsSecure Hash AlgorithmsSymmetric cryptographyMessage authentication codes (MACs)Hash-based MACs (HMACs)Stream ciphersBlock ciphersData Encryption Standard (DES)Advanced Encryption Standard (AES)How AES worksAn OpenSSL example of how to encrypt and decrypt using AESSummary
MathematicsModular arithmeticSetsFieldsFinite fieldsPrime fieldsGroupsAbelian groupsRingsCyclic groupsOrderAsymmetric cryptographyInteger factorizationDiscrete logarithmElliptic curvesPublic and private keysRSAElliptic curve cryptographyMathematics behind ECCThe discrete logarithm problem in ECCRSA using OpenSSLEncryption and decryption using RSAECC using OpenSSLDigital signaturesRSA digital signature algorithmsThe elliptic curve digital signature algorithmHow to generate a digital signature using OpenSSLCryptographic constructs and blockchain technologyHomomorphic encryptionSigncryptionSecret sharingCommitment schemesZero-knowledge proofszk-SNARKszk-STARKsZero-knowledge range proofs—ZKRPsDifferent types of digital signaturesBlind signaturesMultisignaturesThreshold signaturesAggregate signaturesRing signaturesEncoding schemesBase64Base58Applications of cryptographic hash functionsMerkle treesPatricia treesDistributed hash tablesSummary
Introducing the consensus problemThe Byzantine generals problemFault toleranceTypes of fault-tolerant consensusState machine replicationFLP impossibilityLower bounds on the number of processors to solve consensusAnalysis and designModelProcessesTiming assumptionsSynchronyAsynchronyPartial synchronyClassificationAlgorithmsCFT algorithmsPaxosRaftBFT algorithmsPractical Byzantine Fault ToleranceIstanbul Byzantine Fault ToleranceTendermintNakamoto consensusProof of stake (PoS)HotStuffChoosing an algorithmFinalitySpeed, performance, and scalabilitySummary
Bitcoin—an overviewThe beginnings of BitcoinEgalitarianism versus authoritarianismBitcoin definitionBitcoin—A user's perspectiveSending a paymentCryptographic keysPrivate keys in BitcoinPublic keys in BitcoinAddresses in BitcoinBase58Check encodingVanity addressesTransactionsThe transaction lifecycleTransaction poolTransaction feesThe transaction data structureMetadataInputsOutputsVerificationThe Script languageTypes of scriptsContractsCoinbase transactionsTransaction validationTransaction bugsTransaction malleabilityValue overflowBlockchainThe genesis blockStale and orphan blocksSize of the blockchainNetwork difficultyMiningTasks of the minersMining rewardsProof of WorkThe mining algorithmThe hash rateMining systemsCPUGPUFPGAASICsMining poolsSummary
The Bitcoin networkFull client and SPV clientBloom filtersWalletsNon-deterministic walletsDeterministic walletsHierarchical deterministic walletsBrain walletsPaper walletsHardware walletsOnline walletsMobile walletsBitcoin paymentsInnovation in BitcoinBitcoin Improvement ProposalsAdvanced protocolsSegregated WitnessBitcoin CashBitcoin UnlimitedBitcoin GoldBitcoin investment and buying and selling BitcoinSummary
Bitcoin client installationTypes of clients and toolsbitcoindbitcoin-clibitcoin-qtSetting up a Bitcoin nodeSetting up the source codeSetting up bitcoin.confStarting up a node in the testnetStarting up a node in regtestExperimenting further with bitcoin-cliUsing the Bitcoin command-line tool – bitcoin-cliUsing the JSON RPC interface Using the HTTP REST interfaceBitcoin programmingSummary
Introducing altcoinsTheoretical foundationsAlternatives to Proof of WorkProof of StorageProof of Stake (PoS)Various types of stakeProof of Activity (PoA)Non-outsourceable puzzlesDifficulty adjustment and retargeting algorithmsKimoto Gravity WellDark Gravity WaveDigiShieldMIDASBitcoin limitationsPrivacy and anonymityMixing protocolsThird-party mixing protocolsInherent anonymityExtended protocols on top of BitcoinColored coinsCounterpartyDevelopment of altcoinsConsensus algorithmsHashing algorithmsDifficulty adjustment algorithmsInter-block timeBlock rewardsReward halving rateBlock size and transaction sizeInterest rateCoinageTotal supply of coinsToken versus cryptocurrencyInitial Coin Offerings (ICOs)ERC20 standardSummary

HistoryDefinitionRicardian contractsSmart contract templatesOraclesSoftware and network-assisted proofsTLSNotaryTLS-N based mechanismHardware device-assisted proofsAndroid proofLedger proofTrusted hardware-assisted proofsTypes of blockchain oraclesInbound oraclesOutbound oraclesBlockchain oracle servicesDeploying smart contractsThe DAOSummary
Ethereum – an overviewThe yellow paperUseful mathematical symbolsThe Ethereum blockchainEthereum – a user's perspectiveThe Ethereum networkThe mainnetTestnetsPrivate netsComponents of the Ethereum ecosystemKeys and addressesAccountsTypes of accountsTransactions and messagesRLPContract creation transactionsMessage call transactionsMessagesTransaction validation and executionThe transaction substateState storage in the Ethereum blockchainTransaction receiptsEther cryptocurrency/tokens (ETC and ETH)The Ethereum Virtual Machine (EVM)Execution environmentThe machine stateThe iterator functionSmart contractsNative contractsThe elliptic curve public key recovery functionThe SHA-256-bit hash functionThe RIPEMD-160-bit hash function The identity/datacopy functionBig mod exponentiation functionElliptic curve point addition functionElliptic curve scalar multiplicationElliptic curve pairingBlake2 compression function 'F'Summary
Blocks and blockchainThe genesis blockThe block validation mechanismBlock finalizationBlock difficulty mechanismGasFee scheduleWallets and client softwareWalletsGethEthParityTrinityLight clientsInstallation and usageGethEthereum account management using GethHow to query the blockchain using GethEthereum keystoreEth installationOpenEthereum installationMetaMaskInstallationCreating and funding an account using MetaMaskNodes and minersThe consensus mechanismForks in the blockchainEthashCPU miningGPU miningBenchmarkingMining rigsMining poolsASICsAPIs, tools, and DAppsApplications (DApps and DAOs) developed on EthereumToolsGeth JSON RPC APIExamplesSupporting protocolsWhisperSwarmProgramming languagesRuntime bytecodeOpcodesSummary
OverviewTest networksComponents of a private networkNetwork IDThe genesis fileData directoryFlags and their meaningStatic nodesStarting up the private networkMining on the private networkRemix IDEMetaMaskUsing MetaMask and Remix IDE to deploy a smart contractAdding a custom network to MetaMask and connecting Remix IDE with MetaMaskImporting accounts into MetaMask using keystore filesDeploying a contract with MetaMaskInteracting with a contract through MetaMask using Remix IDESummary
LanguagesCompilersThe Solidity compilerInstallationFunctionsTools and librariesNode.jsGanache CLIGanacheFrameworksTruffleDrizzleEmbarkBrownieWaffleEtherlimeOpenZeppelinContract development and deploymentWriting smart contractsTesting smart contractsDeploying smart contractsThe layout of a Solidity source code ﬁleVersion pragmaImportCommentsThe Solidity languageVariablesLocal variablesGlobal variablesState variablesData typesValue typesReference typesControl structuresEventsInheritanceLibrariesFunctionsError handlingSummary
Exploring Web3 with GethContract deploymentPOST requestsRetrieving the list of accountsInteracting with contracts via frontendsThe HTML and JavaScript frontendInstalling Web3.js JavaScript libraryInteracting with contracts via a web frontendCreating an app.js JavaScript fileCreating a Web3 objectCalling contract functionsDevelopment frameworksUsing Truffle to develop a decentralized applicationInstalling and initializing TruffleCompiling, testing, and migrating using TruffleInteracting with the contractUsing Truffle to test and deploy smart contractsDeployment on decentralized storage using IPFSSummary
Ethereum 2.0—an overviewGoalsMain features Roadmap of EthereumDevelopment phasesPhase 0The beacon chainBeacon nodesValidator nodesBeacon and validator node comparisonDeposit contractsFork choiceP2P interface (networking)ETH 2Simple SerializeBLS cryptographyPhase 1Shard chainsTransitioning from Ethereum 1 to Ethereum 2Phase 2 Phase 3Architecture Summary
Projects under HyperledgerDistributed ledgersFabricSawtoothIrohaIndyBesuBurrowLibrariesAriesTransactQuiltUrsaToolsAvalonCelloCaliperExplorerDomain-specificGridLabsHyperledger reference architectureHyperledger design principlesModular structurePrivacy and confidentialityIdentityScalabilityDeterministic transactionsAuditabilityInteroperabilityPortabilityRich data queriesHyperledger FabricMembership servicesBlockchain servicesConsensus servicesDistributed ledgerThe peer-to-peer protocolLedger storageSmart contract servicesAPIs and CLIsComponentsPeersClientsChannelsWorld state databaseTransactionsMembership Service ProviderSmart contractsCrypto service providerApplications on blockchainChaincode implementationThe application modelConsensus in Hyperledger FabricThe transaction lifecycle in Hyperledger FabricFabric 2.0New chaincode lifecycle managementNew chaincode application patternsEnhanced data privacyExternal chaincode launcherRaft consensusBetter performanceHyperledger SawtoothCore featuresModular design Parallel transaction executionGlobal state agreement Dynamic and pluggable consensus algorithmsMulti-language supportEnhanced event mechanismOn-chain governanceInteroperabilityConsensus in SawtoothPoETPBFT RaftTransaction lifecycleComponentsValidatorREST APIClientStateTransaction processorsTransaction familiesREST APISetting up a Sawtooth development environmentPrerequisitesUsing PoETUsing PBFTSetting up a Sawtooth network Summary
Tokenization on a blockchainAdvantages of tokenizationDisadvantages of tokenizationTypes of tokensFungible tokensNon-fungible tokensStable tokensFiat collateralizedCommodity collateralizedCrypto collateralizedAlgorithmically stableSecurity tokensProcess of tokenizationToken offeringsInitial coin offeringsSecurity token offeringsInitial exchange offeringsEquity token offeringsDecentralized autonomous initial coin offeringOther token offeringsToken standardsERC-20ERC-223ERC-777ERC-721ERC-884ERC-1400ERC-1404Trading and financeFinancial marketsTradingExchangesOrders and order propertiesOrder management and routing systemsComponents of a tradeThe underlying instrumentTrade lifecycleOrder anticipatorsMarket manipulationDeFiTrading tokensRegulationBuilding an ERC-20 tokenPre requisitesBuilding the Solidity contractSolidity contract source codeDeploying the contract on the Remix JavaScript virtual machineAdding tokens in MetaMaskEmerging conceptsTokenomics/token economicsToken engineeringToken taxonomySummary
The Internet of ThingsInternet of Things architecturePhysical object layerDevice layerNetwork layerManagement layerApplication layerBenefits of IoT and blockchain convergence Implementing blockchain-based IoT in practiceSetting up Raspberry PiSetting up the first node Setting up the Raspberry Pi node Building the electronic circuitGovernmentBorder controlVotingCitizen identification (ID cards)HealthFinanceInsurancePost-trade settlementFinancial crime preventionPaymentsDecentralizationFaster settlementBetter resilienceCross-border paymentsPeer-to-peer loansMediaSummary
Enterprise solutions and blockchainSuccess factorsLimiting factorsSlow performanceLack of access governanceLack of privacyProbabilistic consensusTransaction feesRequirementsPrivacyConfidentialityAnonymityPerformanceScalability/speedAccess governanceFurther requirementsComplianceInteroperableIntegrationEase of useMonitoringSecure off-chain computationBetter toolsEnterprise blockchain versus public blockchainUse cases of enterprise blockchainsEnterprise blockchain architectureNetwork layerProtocol layerPrivacy layerGovernance layerIntegration layerWhat is Apache Camel?Application layerSecurity, performance, scalability, monitoringDesigning enterprise blockchain solutionsTOGAFBusiness architecture domainData architecture domainApplication architecture domainTechnology architecture domainArchitecture development methodPreliminary phaseArchitecture visionBusiness architectureInformation systems architectureTechnology architectureOpportunities and solutionsMigration planningImplementation governanceArchitecture change managementBlockchain in the cloudCurrently available enterprise blockchainsCordaQuorumFabricAutonityComparison of main platformsEnterprise blockchain challengesInteroperabilityLack of standardizationComplianceBusiness challengesCordaArchitectureCorda networkState objectsTransactionsConsensusFlowsCorDappsComponentsNodesThe permissioning serviceNetwork map serviceNotary serviceOracle serviceTransactionsVaultsOther toolsTransaction flowCorda development environmentQuorumArchitectureEnhanced P2PEnhanced state (private and public)Pluggable consensusNo transaction feesPrivate transactionsModified block generation mechanismModified block validation mechanismEnhanced RPC APIPrivacy managerTransaction managerEnclaveCryptography used in QuorumPrivacyEnclave encryptionTransaction propagation to transaction managersEnclave decryptionAccess control with permissioningPerformancePluggable consensusSetting up Quorum with IBFTQuorum WizardInstalling Quorum WizardRunning Quorum Wizard to create a new networkCakeshopRunning a private transactionNode 1Node 2Node 3Node 4Viewing the transaction in CakeshopFurther investigationNode 1Node 2, which is privy to the transactionNode 3, which is not privy to the transactionOther Quorum projectsRemix pluginPluggable architectureSummary
ScalabilityBlockchain planesNetwork planeConsensus planeStorage planeView planeSide planeMethods for improving scalabilityLayer 0 – network solutionsLayer 1 – on-chain solutionsLayer 2 – off-chain and multichain solutionsPrivacyAnonymityConfidentialityTechniques to achieve privacyLayer 0Layers 1 and 2SecurityFormal verificationModel checkingVerifying consensus mechanismsSmart contract securityStatic analysis in Remix IDEWhy3OyenteOther toolsFormal verification of smart contractsOther challengesInteroperabilityPolkadotLack of standardizationPost-quantum resistanceCompliance and regulationSummary
Emerging trendsNew implementations of blockchain technologyApplication-specific blockchains Start-upsTechnology improvementsStandardizationConsortiaEnhancementsOngoing research and studyCryptographyCryptoeconomicsHardware developmentFormal methods and securityNew programming languagesEducation and employment within blockchain Innovative blockchain applicationsBlockchain as a ServiceConvergence with other technologiesAlternatives to blockchainsSome debatable ideasPublic versus private on the blockchainCentral bank digital currency Areas to addressRegulationIllegal activityPrivacy or transparencyBlockchain research topicsSmart contractsCryptographic function limitationsConsensus algorithmsScalabilityCode obfuscationBlockchain and AIThe future of blockchainSummary

Content preview from Mastering Blockchain - Third Edition

5 Consensus Algorithms

Consensus is a fundamental problem in distributed systems. Since the 1970s this problem has been researched in the context of distributed systems, but recently, with the advent of blockchain technology, a renewed interest has arisen in developing distributed consensus algorithms that are suitable for blockchain networks. In this chapter, we will explore the underlying techniques behind distributed consensus algorithms, their inner workings, and new algorithms that have been specifically developed for blockchain networks.

In addition, we will introduce various well-known algorithms in a traditional distributed systems arena that can also be implemented in blockchain networks with some modifications, such as Paxos, Raft, ...