book

HBase: The Definitive Guide

by Lars George

September 2011

Intermediate to advanced

552 pages

16h 31m

English

O'Reilly Media, Inc.

Read now

Unlock full access

General InformationHBase VersionBuilding the ExamplesHush: The HBase URL ShortenerRunning HushConventions Used in This BookUsing Code ExamplesSafari® Books OnlineHow to Contact UsAcknowledgments
The Dawn of Big DataThe Problem with Relational Database SystemsNonrelational Database Systems, Not-Only SQL or NoSQL?DimensionsScalabilityDatabase (De-)NormalizationBuilding BlocksBackdropTables, Rows, Columns, and CellsAuto-ShardingStorage APIImplementationSummaryHBase: The Hadoop DatabaseHistoryNomenclatureSummary
Quick-Start GuideRequirementsHardwareServersNetworkingSoftwareOperating systemFilesystemJavaHadoopSSHDomain Name ServiceSynchronized timeFile handles and process limitsDatanode handlersSwappinessWindowsFilesystems for HBaseLocalHDFSS3Other FilesystemsInstallation ChoicesApache Binary ReleaseBuilding from SourceRun ModesStandalone ModeDistributed ModePseudodistributed modeFully distributed modeSpecifying region serversZooKeeper setupUsing the existing ZooKeeper ensembleConfigurationhbase-site.xml and hbase-default.xmlhbase-env.shregionserverlog4j.propertiesExample Configurationhbase-site.xmlregionservershbase-env.shClient ConfigurationDeploymentScript-BasedApache WhirrPuppet and ChefOperating a ClusterRunning and Confirming Your InstallationWeb-based UI IntroductionShell IntroductionStopping the Cluster
General NotesCRUD OperationsPut MethodSingle PutsThe KeyValue classClient-side write bufferList of PutsAtomic compare-and-setGet MethodSingle GetsThe Result classList of GetsRelated retrieval methodsDelete MethodSingle DeletesList of DeletesAtomic compare-and-deleteBatch OperationsRow LocksScansIntroductionThe ResultScanner ClassCaching Versus BatchingMiscellaneous FeaturesThe HTable Utility MethodsThe Bytes Class
FiltersIntroduction to FiltersThe filter hierarchyComparison operatorsComparatorsComparison FiltersRowFilterFamilyFilterQualifierFilterValueFilterDependentColumnFilterDedicated FiltersSingleColumnValueFilterSingleColumnValueExcludeFilterPrefixFilterPageFilterKeyOnlyFilterFirstKeyOnlyFilterInclusiveStopFilterTimestampsFilterColumnCountGetFilterColumnPaginationFilterColumnPrefixFilterRandomRowFilterDecorating FiltersSkipFilterWhileMatchFilterFilterListCustom FiltersFilters SummaryCountersIntroduction to CountersSingle CountersMultiple CountersCoprocessorsIntroduction to CoprocessorsThe Coprocessor ClassCoprocessor LoadingLoading from the configurationLoading from the table descriptorThe RegionObserver ClassHandling region life-cycle eventsState: pending openState: openState: pending closeHandling client API eventsThe RegionCoprocessorEnvironment classThe ObserverContext classThe BaseRegionObserver classThe MasterObserver ClassThe MasterCoprocessorEnvironment classThe BaseMasterObserver classEndpointsThe CoprocessorProtocol interfaceThe BaseEndpointCoprocessor classHTablePoolConnection Handling
Schema DefinitionTablesTable PropertiesColumn FamiliesHBaseAdminBasic OperationsTable OperationsSchema OperationsCluster OperationsCluster Status Information
Introduction to REST, Thrift, and AvroInteractive ClientsNative JavaRESTOperationSupported formatsPlain (text/plain)XML (text/xml)JSON (application/json)Protocol Buffer (application/x-protobuf)Raw binary (application/octet-stream)REST Java clientThriftInstallationOperationExample: PHPAvroInstallationOperationOther ClientsBatch ClientsMapReduceNative JavaClojureHivePigCascadingShellBasicsCommandsGeneralData definitionData manipulationToolsReplicationScriptingWeb-based UIMaster UIMain pageUser Table pageZooKeeper pageRegion Server UIMain pageShared Pages
FrameworkMapReduce IntroductionClassesInputFormatMapperReducerOutputFormatSupporting ClassesMapReduce LocalityTable SplitsMapReduce over HBasePreparationStatic ProvisioningDynamic ProvisioningData SinkData SourceData Source and SinkCustom Processing

Seek Versus TransferB+ TreesLog-Structured Merge-TreesStorageOverviewWrite PathFilesRoot-level filesTable-level filesRegion-level filesRegion splitsCompactionsHFile FormatKeyValue FormatWrite-Ahead LogOverviewHLog ClassHLogKey ClassWALEdit ClassLogSyncer ClassLogRoller ClassReplaySingle logLog splittingEdits recoveryDurabilityRead PathRegion LookupsThe Region Life CycleZooKeeperReplicationLife of a Log EditNormal processingNon-Responding slave clustersInternalsChoosing region servers to replicate toKeeping track of logsReading, filtering, and sending editsCleaning logsRegion server failover
Key DesignConceptsTall-Narrow Versus Flat-Wide TablesPartial Key ScansPaginationTime Series DataTime-Ordered RelationsAdvanced SchemasSecondary IndexesSearch IntegrationTransactionsBloom FiltersVersioningImplicit VersioningCustom Versioning
IntroductionThe Metrics FrameworkContexts, Records, and MetricsMaster MetricsRegion Server MetricsRPC MetricsJVM MetricsInfo MetricsGangliaInstallationGanglia-related stepsGanglia monitoring daemonGanglia meta daemonGanglia web frontendHBase-related stepsUsageJMXJConsoleJMX Remote APINagios
Garbage Collection TuningMemstore-Local Allocation BufferCompressionAvailable CodecsSnappyLZOGZIPVerifying InstallationCompression test toolStartup checkEnabling CompressionOptimizing Splits and CompactionsManaged SplittingRegion HotspottingPresplitting RegionsLoad BalancingMerging RegionsClient API: Best PracticesConfigurationLoad TestsPerformance EvaluationYCSB
Operational TasksNode DecommissioningRolling RestartsAdding ServersPseudodistributed modeAdding a local backup masterAdding a local region serverFully distributed clusterAdding a backup masterAdding a region serverData TasksImport and Export ToolsCopyTable ToolBulk ImportBulk load procedureUsing the importtsv toolUsing the completebulkload ToolAdvanced usageReplicationAdditional TasksCoexisting ClustersRequired PortsChanging Logging LevelsTroubleshootingHBase FsckAnalyzing the LogsCommon IssuesBasic setup checklistFile handlesDataNode connectionsCompressionGarbage collection/memory tuningStability issuesZooKeeper problems“Could not obtain block” errors
HBase 0.92.0HBase 0.94.0
Upgrading to HBase 0.90.xFrom 0.20.x or 0.89.xWithin 0.90.xUpgrading to HBase 0.92.0
Cloudera’s Distribution Including Apache Hadoop