book

Implementing an IBM InfoSphere BigInsights Cluster using Linux on Power

by Dino Quintero, Esteban Arias Navarro, Pablo Barquero Garro, Rodrigo Ceron Ferreira de Castro, Luis Carlos Cruz Huertas, Peng Jiang, Franz Friedrich Liebinger Portela, Peter McCullagh, Ichsan Mulia Permata, Joanna Wong, John Wright

June 2015

Intermediate to advanced

236 pages

6h 11m

English

IBM Redbooks

Read now

Unlock full access

Trademarks
AuthorsNow you can become a published author, too!Comments welcomeStay connected to IBM Redbooks
1.1 InfoSphere BigInsights1.1.1 The advantages of InfoSphere BigInsights1.1.2 What practical problems can be solved with InfoSphere BigInsights1.2 Linux on IBM Power Systems1.2.1 IBM Power Systems1.2.2 Linux for big data1.3 IBM Platform Symphony1.4 IBM Spectrum Scale-FPO (formerly GPFS-FPO)1.5 IBM Platform Cluster Manager
2.1 The big data environment reference architecture2.2 Hardware architecture for InfoSphere BigInsights clusters on POWER2.2.1 General architecture2.2.2 The POSHv2 architecture2.3 Software architecture for InfoSphere BigInsights clusters2.3.1 Cluster management software
3.1 Linux on Power Systems3.1.1 Operating system installation3.1.2 Operating system prerequisites setup for InfoSphere BigInsights3.2 InfoSphere BigInsights3.2.1 Installation3.3 Platform Cluster Manager - Advanced Edition3.3.1 Planning a system configuration3.3.2 Performing the installation3.3.3 Managing and configuring the IBM Power Systems nodes3.3.4 Provisioning templates3.3.5 Adding nodes to Platform Cluster Manager - Advanced Edition
4.1 Important factors for sizing an InfoSphere BigInsights cluster4.1.1 Scalability4.1.2 Availability4.2 Customizing the predefined configurations4.3 IBM Spectrum Scale (formerly GPFS) considerations4.4 High availability considerations4.4.1 Designing for high availability4.5 Throughput and bandwidth considerations4.5.1 The data network4.5.2 Administrative/management network4.6 Data volumes considerations4.7 Security, user authentication, and edge nodes4.7.1 Security preferred practices in non-relational data stores4.7.2 Securing IBM Spectrum Scale4.7.3 Securing data storage and transaction logs4.8 Impact of use cases in design4.8.1 Workload characteristics definition
5.1 IBM Elastic Storage Server5.2 IBM Platform Symphony MapReduce5.3 File system: Spectrum Scale / HDFS (architectural changes when not using Spectrum Scale)5.4 InfoSphere BigInsights high availability5.4.1 IBM Spectrum Scale5.4.2 HDFS5.5 Security: InfoSphere BigInsights user authentication5.5.1 Using flat file security5.5.2 Using LDAP security
6.1 Managing nodes in a Platform Cluster Manager - Advanced Edition environment6.1.1 Adding nodes to a Platform Cluster Manager - Advanced Edition environment6.1.2 Removing nodes from a Platform Cluster Manager - Advanced Edition environment (including the monitored node entry)6.1.3 Monitoring a Platform Cluster Manager - Advanced Edition managed node6.2 Managing InfoSphere BigInsights cluster nodes within Platform Cluster Manager - Advanced Edition6.2.1 Creating a cluster template6.2.2 Creating a cluster from a cluster template

7.1 Tuning IBM InfoSphere BigInsights7.1.1 Tuning at the operating system level7.1.2 Tuning Hadoop7.1.3 Tuning BigSQL7.2 Tuning IBM Spectrum Scale (formerly GPFS)7.2.1 Tuning at the operating system level7.2.2 Tuning the Spectrum Scale daemon (formerly the GPFS daemon)7.2.3 Configuring the Spectrum Scale file system7.3 Tuning the Platform Symphony MapReduce framework
Test cluster descriptionInstallation and configuration of IBM Spectrum Scale (formerly GPFS)Configuration of the IBM Spectrum Scale File Placement Optimizer Hadoop ConnectorInstalling and configuring IBM Java and Apache HadoopRunning a Hadoop MapReduce jobInstalling and configuring IBM Platform Symphony V7.1Running Hadoop MapReduce jobs on Platform Symphony
Postscripts that are defined in the InfoSphere BigInsights cluster templatePostscripts that are defined in the image profile
BigData Enablement and Administration Toolkit overviewBig data solution deployment with BigData Enablement and Administration Toolkit
IBM RedbooksOnline resourcesHelp from IBM

Content preview from Implementing an IBM InfoSphere BigInsights Cluster using Linux on Power

Introduction to the solution

This book provides information about how to deploy an IBM InfoSphere BigInsights cluster on IBM Power Systems servers running Linux. It also provides information about how to manage the clustered solution, including automated deployment by using IBM Platform Cluster Manager.

This chapter covers the following topics:

•InfoSphere BigInsights

•Linux on IBM Power Systems

•IBM Platform Symphony

•IBM Spectrum Scale-FPO (formerly GPFS-FPO)

•IBM Platform Cluster Manager

After reading this chapter, you will understand the advantages ...