book

Real-World Hadoop

by Ellen Friedman, Ted Dunning

April 2015

Beginner to intermediate

104 pages

2h 28m

English

O'Reilly Media, Inc.

Read now

Unlock full access

How to Use This Book
A Day in the Life of a Big Data ProjectFrom Aadhaar to Your Own Big Data ProjectWhat Hadoop and NoSQL DoWhen Are Hadoop and NoSQL the Right Choice?
Typical FunctionsData Storage and PersistenceData IngestApache KafkaApache SqoopApache FlumeData Extraction from HadoopProcessing, Transforming, QueryingStreamingMicro-batchingBatch ProcessingInteractive QueryImpalaApache DrillApache SparkSearch Abuse—Using Search and Indexing for Interactive QueryVisualization ToolsIntegration via ODBC and JDBC
Use of Existing Non-Hadoop ApplicationsMaking Use of a Realtime Distributed File SystemMeeting SLAsDeploying Data at Scale to Remote LocationsConsistent Data VersioningFinding the Keys to Success
Tip #1: Pick One Thing to Do FirstTip #2: Shift Your ThinkingTip #3: Start Conservatively But Plan to ExpandTip #4: Be Honest with YourselfTip #5: Plan Ahead for MaintenanceTip #6: Think Big: Don’t Underestimate What You Can (and Will) Want to DoTip #7: Explore New Data FormatsTip #8: Consider Data Placement When You Expand a ClusterTip #9: Plot Your ExpansionTip #10: Form a Queue to the Right, PleaseTip #11: Provide Reliable Primary Persistence When Using Search ToolsTip #12: Establish Remote Clusters for Disaster RecoveryTip #13: Take a Complete View of PerformanceTip #14: Read Our Other Books (Really!)Tip # 15: Just Do It
Data Warehouse OptimizationData HubCustomer 360Recommendation EngineMarketing OptimizationLarge Object StoreLog ProcessingRealtime AnalyticsTime Series Database
TelecomsWhat Customers WantWorking with MoneySensor Data, Predictive Maintenance, and a “Time Machine”A Time MachineManufacturingExtending Quality Assurance
Additional Publications

Content preview from Real-World Hadoop

About the Authors

Ted Dunning is Chief Applications Architect at MapR Technologies and active in the open source community, being a committer and PMC member of the Apache Mahout, Apache ZooKeeper, and Apache Drill projects, and serving as a mentor for the Storm, Flink, Optiq, and Datafu Apache incubator projects. He has contributed to Mahout clustering, classification, matrix decomposition algorithms, and the new Mahout Math library, and recently designed the t-digest algorithm used in several open source projects.Ted was the chief architect behind the MusicMatch (now Yahoo Music) and Veoh recommendation systems, built fraud-detection systems for ID Analytics (LifeLock), and has 24 issued patents to date. Ted has a PhD in computing science from University of Sheffield. When he’s not doing data science, he plays guitar and mandolin. Ted is on Twitter at @ted_dunning.

Ellen Friedman is a solutions consultant and well-known speaker andauthor, currently writing mainly about big data topics. She is a committerfor the Apache Mahout project and a contributor to the ApacheDrill project. With a PhD in Biochemistry from Rice University, shehas years of experience as a research scientist and has written about avariety of technical topics including molecular biology, nontraditionalinheritance, oceanography, and large-scale computing. Ellen is alsoco-author of a book of magic-themed cartoons, A Rabbit Under theHat. Ellen is on Twitter at @Ellen_Friedman.