book

IBM InfoSphere Streams: Accelerating Deployments with Analytic Accelerators

Name: IBM InfoSphere Streams: Accelerating Deployments with Analytic Accelerators
ISBN: 0738439193

by Chuck Ballard, Oliver Brandt, Bharath Devaraju, Daniel Farrell, Kevin Foster, Chris Howard, Peter Nicholls, Ankit Pasricha, Roger Rea, Norbert Schulz, Tetsuya Shimada, John Thorson, Sandra Tucker, Robert Uleman

February 2014

Beginner to intermediate

556 pages

12h 19m

English

IBM Redbooks

Read now

Unlock full access

Front cover
Notices
Trademarks
Preface
AuthorsNow you can become a published author, too!Comments welcomeStay connected to IBM Redbooks
Chapter 1. Introduction
1.1 The challenge of getting started1.1.1 Accelerators and toolkits1.2 Using this book
Chapter 2. Application programming using Streams Studio
2.1 SPL Graphical Editor2.1.1 Adding operators to the graph2.1.2 Connecting operators2.1.3 Layout options and Outline view2.1.4 Defining global types and schemas2.1.5 Editing operator details2.1.6 Generating SPL code2.2 Application development use cases2.2.1 Design: Sketching an application2.2.2 Implementing an application2.2.3 Composites: Reusable subgraphs2.2.4 Making changes: Refactoring2.2.5 Program understanding
Chapter 3. Visualizing stream data
3.1 Stream data, views, and charts3.2 How data visualization works3.3 Use cases3.3.1 Debugging using Streams Studio3.3.2 Application monitoring using Streams Console3.3.3 Hints and tips
Chapter 4. Analytics entirely with SPL
4.1 Volume, variety, and velocity4.2 Model view controller (MVC), and Streams4.3 Flow rate sensor: Example application we create4.3.1 Sample output from our flow rate sensor4.3.2 Flow rate sensor: First ingest, heartbeat generator4.3.3 Flow rate sensor: Actual ingest, parsing semi-structured data4.3.4 Flow rate sensor: React phase (simulated)4.3.5 Flow rate sensor: User defined functions, (read-only) maps4.3.6 Flow rate sensor: Analyze, read-write maps, windows, other4.4 Conclusion, how to proceed
Chapter 5. Streams and DataStage integration
5.1 Introduction to Streams processes5.2 Runtime architecture5.3 Metadata integration5.3.1 Integration Setup Overview5.4 Sample application5.4.1 Importing Streams certificate to DataStage5.4.2 Streams-to-DataStage application5.5 DataStage Job design practices
Chapter 6. Streams integration with IBM BigInsights
6.1 Streams and big data challenges6.1.1 Application scenarios6.1.2 Large scale data ingest6.1.3 Bootstrap and enrichment6.1.4 Adaptive analytics model6.1.5 Complex social and entity related analysis6.1.6 Application development6.1.7 Application interactions6.1.8 Enabling components6.2 BigInsights summary
Chapter 7. Complex event processing
7.1 The role of the CEP Toolkit7.1.1 Adding the CEP Toolkit to your build path7.2 Stock price watch example7.2.1 First iteration of stock price watch application7.2.2 Second iteration of stock price watch application7.2.3 Third iteration of stock price watch application

Chapter 8. WebSphere MQ, XMSSource, XMSSink
8.1 WebSphere MQ Server, Message Service Client installation8.2 Making the WebSphere MQ resident objects8.3 Setting Streams environment variables, and adding toolkits8.4 The two Streams applications8.5 Using JMS adapters from the Messaging Toolkit8.5.1 Example use cases for the JMS adapters8.5.2 Installing and configuring of WebSphere MQ8.5.3 Installing and configuring for Apache ActiveMQ8.5.4 Compiling and running sample applications8.5.5 Sample applications8.5.6 Verifying the results
Chapter 9. XML, XMLParse, XPath, and xquery
9.1 Scenario 1: Flat, single-tier XML, XMLParse9.2 Scenario 2: Multitiered (list data)9.2.1 Second iteration, XMLParse9.2.2 Third iteration, XMLParse9.2.3 Fourth (final) iteration, XMLParse9.3 The spl-schema-from-xml utility, generating XMLParse code9.4 The xquery() function, including filter9.5 CDATA: Topic that is not covered in detail
Chapter 10. Geospatial Toolkit
10.1 Concepts10.1.1 Moving objects and location-based services10.1.2 Geospatial concepts and operations10.2 Toolkit organization10.2.1 Namespaces10.2.2 Types and enumerations10.2.3 Constructor functions10.2.4 Accessor functions10.2.5 Spatial production functions10.2.6 Spatial relationship functions10.2.7 Validation of input arguments10.2.8 Metric conversion functions10.3 A location-based scenario: tracking vehicles10.3.1 A vehicle simulator10.3.2 Geofencing: detecting entry and exit10.3.3 Predictive geospatial analytics10.4 Conclusion
Chapter 11. TimeSeries Toolkit
11.1 Basics of time series analysis11.1.1 Time series patterns11.1.2 Detecting patterns using the TimeSeries Toolkit11.2 Time series representation and operators overview11.2.1 Time series representation11.2.2 Control signals11.2.3 Types of operators and overview11.3 Preprocessing operators11.3.1 ReSample operator11.3.2 TSWindowing operator11.3.3 IncrementalInterpolate operator11.4 Analysis operators11.4.1 DSPFilter operator11.4.2 Fast Fourier Transform11.4.3 Discrete Wavelet Transform operator11.4.4 Seasonal Trend Decomposition operator11.4.5 CrossCorrelate operator11.4.6 Normalize operator11.4.7 FunctionEvaluator operator11.4.8 Distribution operator11.5 Modeling operators11.5.1 HoltWinters operator11.5.2 ARIMA operator11.5.3 FMPFilter operator11.5.4 Kalman operator11.5.5 GAM operators11.5.6 GMM operator11.5.7 LPC operator11.5.8 VAR operator11.5.9 RLSFilter operator11.6 Time series functions11.6.1 The generator functions11.6.2 The Crosscorrelate function11.6.3 The convolve function11.6.4 The rms function
Chapter 12. Developing Java primitive operators
12.1 Operator lifecycle12.2 Threading in a Java operator12.3 Creating a simple operator12.3.1 Operator code layout12.3.2 Defining the operator model12.3.3 Implementing the operator in Java12.3.4 Compiling the operator code12.3.5 Testing the operator code12.3.6 Creating an SPL application12.3.7 Adding custom metrics12.3.8 Implementing a tuple consumer operator12.4 Java development using Streams Studio
Chapter 13. Text Analytics, AQL
13.1 Overview text analytics, by example13.2 Installing and configuring text analytics tools13.2.1 Installing text analytics tools from Streams Studio installation13.2.2 Installing text analytics tools from a BigInsights install13.2.3 Configuring your Streams project, add a BigInsights project13.3 First AQL example, Apache HTTP log file13.3.1 Improving the first example, Apache HTTP log file13.4 Guided team-based AQL and using AQL tools13.5 Regular expressions (regex)13.5.1 AQL tools to aid in regex development and debug13.5.2 Regex directly inside Streams; no AQL required13.6 Additional AQL objects and techniques13.6.1 The kitchen sink AQL script and Streams application13.7 Topics not covered
Chapter 14. IBM Accelerator for Telecommunications Event Data Analytics V1.2
14.1 Overview of TEDA14.2 Installing TEDA14.3 Understanding concepts and terms14.3.1 Application components14.3.2 Application infrastructure14.3.3 Configuration14.3.4 Fault tolerance14.4 Customizing TEDA14.4.1 Workflow overview14.4.2 Preparation14.4.3 Defining the sample use case14.4.4 Exercise 1: Basic setup14.4.5 Exercise 2: Writing to the database14.4.6 Exercise 3: Adding an aggregation operator14.5 Conclusion
Chapter 15. SPSS Toolkit
15.1 An overview of InfoSphere Streams and SPSS15.1.1 Integrating InfoSphere Streams and SPSS15.1.2 Roles and terminology15.1.3 Example development process15.2 Coordinating Data Analyst and Streams developer efforts15.3 Building the predictive models15.4 Configuring the SPSSScoring operator15.5 Summary
Appendix A. Additional material
Locating the web materialUsing the web material
Related publications
IBM RedbooksOnline resourcesHelp from IBM
Back cover

Overview

This IBM® Redbooks® publication describes visual development, visualization, adapters, analytics, and accelerators for IBM InfoSphere® Streams (V3), a key component of the IBM Big Data platform. Streams was designed to analyze data in motion, and can perform analysis on incredibly high volumes with high velocity, using a wide variety of analytic functions and data types.

The Visual Development environment extends Streams Studio with drag-and-drop development, provides round tripping with existing text editors, and is ideal for rapid prototyping. Adapters facilitate getting data in and out of Streams, and V3 supports WebSphere MQ, Apache Hadoop Distributed File System, and IBM InfoSphere DataStage. Significant analytics include the native Streams Processing Language, SPSS Modeler analytics, Complex Event Processing, TimeSeries Toolkit for machine learning and predictive analytics, Geospatial Toolkit for location-based applications, and Annotation Query Language for natural language processing applications. Accelerators for Social Media Analysis and Telecommunications Event Data Analysis sample programs can be modified to build production level applications.

Want to learn how to analyze high volumes of streaming data or implement systems requiring high performance across nodes in a cluster? Then this book is for you.

Please note that the additional material referenced in the text is not available from IBM.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

IBM InfoSphere DataStage Data Flow and Job Design

Publisher Resources

ISBN: 0738439193Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills