book

IBM SPSS Modeler Cookbook

Name: IBM SPSS Modeler Cookbook
ISBN: 9781849685467

by Keith McCormick, Dean Abbott

October 2013

Beginner to intermediate

382 pages

9h 55m

English

Packt Publishing

Read now

Unlock full access

IBM SPSS Modeler Cookbook
Table of Contents
IBM SPSS Modeler Cookbook
Credits
Foreword
About the Authors
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and moreWhy Subscribe?Free Access for Packt account holdersInstant Updates on New Packt Books
Preface
What is CRISP-DM?
Data mining is a business process

The IBM SPSS Modeler workbench
A brief history of the Clementine workbench
Historical introduction to scripting
What this book covers
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions
1. Data Understanding
Introduction
Using an empty aggregate to evaluate sample size
Getting readyHow to do it...How it works...There's more...A modified versionSee also
Evaluating the need to sample from the initial data
Getting readyHow to do it...How it works...There's more...See also
Using CHAID stumps when interviewing an SME
Getting readyHow to do it...How it works...See also
Using a single cluster K-means as an alternative to anomaly detection
Getting readyHow to do it...How it works...There's more...
Using an @NULL multiple Derive to explore missing data
Getting readyHow to do it...How it works...See also
Creating an Outlier report to give to SMEs
Getting readyHow to do it...How it works...See also
Detecting potential model instability early using the Partition node and Feature Selection node
Getting readyHow to do it...How it works...See also
2. Data Preparation – Select
Introduction
Using the Feature Selection node creatively to remove or decapitate perfect predictors
Getting readyHow to do it...How it works...There's more...See also
Running a Statistics node on anti-join to evaluate the potential missing data
Getting readyHow to do it...How it works...See also
Evaluating the use of sampling for speed
Getting readyHow to do it...How it works...There's more...See also
Removing redundant variables using correlation matrices
Getting readyHow to do it...How it works...There's more...See also
Selecting variables using the CHAID Modeling node
Getting readyHow to do it...How it works...There's more...See also
Selecting variables using the Means node
Getting readyHow to do it...How it works...There's more...See also
Selecting variables using single-antecedent Association Rules
Getting readyHow to do it...How it works...There's more...See also
3. Data Preparation – Clean
Introduction
Binning scale variables to address missing data
Getting readyHow to do it...How it works...See also
Using a full data model/partial data model approach to address missing data
Getting readyHow to do it...How it works...There's more...See also
Imputing in-stream mean or median
Getting readyHow to do it...How it works...There's more...See also
Imputing missing values randomly from uniform or normal distributions
Getting readyHow to do it...How it works...There's more...See also
Using random imputation to match a variable's distribution
Getting readyHow to do it...How it works...There's more...See also
Searching for similar records using a Neural Network for inexact matching
Getting readyHow to do it...How it works...There's more...See also
Using neuro-fuzzy searching to find similar names
Getting readyHow to do it...How it works...There's more...See also
Producing longer Soundex codes
Getting readyHow to do it...How it works...There's more...See also
4. Data Preparation – Construct
Introduction
Building transformations with multiple Derive nodes
Getting readyHow to do it...How it works...There's more...
Calculating and comparing conversion rates
Getting readyHow to do it...How it works...There's more...See also
Grouping categorical values
Getting readyHow to do it...How it works...There's more...
Transforming high skew and kurtosis variables with a multiple Derive node
Getting readyHow to do it...How it works...There's more...
Creating flag variables for aggregation
Getting readyHow to do it...How it works...There's more...
Using Association Rules for interaction detection/feature creation
Getting readyHow to do it...How it works...There's more...
Creating time-aligned cohorts
Getting readyHow to do it...How it works...There's more...
5. Data Preparation – Integrate and Format
Introduction
Speeding up merge with caching and optimization settings
Getting readyHow to do it...How it works...See also
Merging a lookup table
Getting readyHow to do it...How it works...See also
Shuffle-down (nonstandard aggregation)
Getting readyHow to do it...How it works...There's more...See also
Cartesian product merge using key-less merge by key
Getting readyHow to do it...How it works...There's more...See also
Multiplying out using Cartesian product merge, user source, and derive dummy
Getting readyHow to do it...How it works...There's more...See also
Changing large numbers of variable names without scripting
Getting readyHow to do it...How it works...There's more...See also
Parsing nonstandard dates
Getting readyHow to do it...How it works...There's more...Nesting functions into one Derive nodePerforming clean downstream of a calculation using a Filter nodeUsing parameters instead of constants in calculationsSee also
Parsing and performing a conversion on a complex stream
Getting readyHow to do it...How it works...See also
Sequence processing
Getting readyHow to do it...How it works...There's more...See also
6. Selecting and Building a Model
Introduction
Evaluating balancing with Auto Classifier
Getting readyHow to do it...How it works...See also
Building models with and without outliers
Getting readyHow to do it...How it works...See also
Using Neural Network for Feature Selection
Getting readyHow to do it...How it works...There's more...See also
Creating a bootstrap sample
Getting readyHow to do it...How it works...There's more...See also
Creating bagged logistic regression models
Getting readyHow to do it...How it works...There's more...See also
Using KNN to match similar cases
Getting readyHow to do it...How it works...See also
Using Auto Classifier to tune models
Getting readyHow to do it...How it works...See also
Next-Best-Offer for large datasets
Getting readyHow to do it...How it works...There's more...See also
7. Modeling – Assessment, Evaluation, Deployment, and Monitoring
Introduction
How (and why) to validate as well as test
Getting readyHow to do it...How it works...See also
Using classification trees to explore the predictions of a Neural Network
Getting readyHow to do it...How it works...See also
Correcting a confusion matrix for an imbalanced target variable by incorporating priors
Getting readyHow to do it...How it works...There's more...See also
Using aggregate to write cluster centers to Excel for conditional formatting
Getting readyHow to do it...How it works...See also
Creating a classification tree financial summary using aggregate and an Excel Export node
Getting readyHow to do it...How it works...See also
Reformatting data for reporting with a Transpose node
Getting readyHow to do it...How it works...There's more...See also
Changing formatting of fields in a Table node
Getting readyHow to do it...How it works...There's more...See also
Combining generated filters
Getting readyHow to do it...How it works...There's more...See also
8. CLEM Scripting
IntroductionCLEM scripting best practicesCLEM scripting shortcomings
Building iterative Neural Network forecasts
Getting readyHow to do it...How it works...Script section 1There's more...
Quantifying variable importance with Monte Carlo simulation
Getting readyHow to do it...How it works...Script section 1Script section 2There's more...
Implementing champion/challenger model management
Getting readyHow to do it...How it works...Script section 1Script section 2There's more...
Detecting outliers with the jackknife method
Getting readyHow to do it...How it works...Script section 1Script section 2Script section 3There's more...
Optimizing K-means cluster solutions
Getting readyHow to do it...How it works...Script section 1Script section 2Script section 3Script section 4There's more...
Automating time series forecasts
Getting readyHow to do it...How it works...Script section 1Script section 2There's more...
Automating HTML reports and graphs
Getting readyHow to do it...How it works...Script section 1Script section 2Script section 3There's more...
Rolling your own modeling algorithm – Weibull analysis
Getting readyHow to do it...How it works...Script section 1There's more...
A. Business Understanding
IntroductionWhat decisions are you trying to make using data?
Define business objectives by Tom Khabaza
The importance of business objectives in data miningDefining the business objectives of a data mining projectUnderstanding the goals of the businessUnderstanding the objectives of your clientConnecting specific objectives to analytical resultsSpecifying your data mining goals
Assessing the situation by Meta Brown
Taking inventory of resourcesReviewing requirements, assumptions, and constraintsIdentifying risks and defining contingenciesDefining terminologyEvaluating costs and benefits
Translating your business objective into a data mining objective by Dean Abbott
The key to the translation – specifying target variablesData mining success criteria – measuring how good the models actually areSuccess criteria for classificationSuccess criteria for estimationOther customized success criteria
Produce a project plan – ensuring a realistic timeline by Keith McCormick
Business understandingData understandingData preparationModelingEvaluationDeployment
Index

Content preview from IBM SPSS Modeler Cookbook

Chapter 3. Data Preparation – Clean

In this chapter, we will cover:

Binning scale variables to address missing data
Using a full data model/partial data model approach to address missing data
Imputing in-stream mean or median
Imputing missing values randomly from uniform or normal distributions
Using random imputation to match a variable's distribution
Searching for similar records using a Neural Network for inexact matching
Using neuro-fuzzy searching to find similar names
Producing longer Soundex codes

Introduction

This chapter addresses the clean subtask of the data preparation phase. CRISP-DM describes this subtask in the following way:

Raise the data quality to the level required by the selected analysis techniques. This may involve selection of clean ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781849685467Other

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

IBM SPSS Modeler Cookbook

by Keith McCormick, Dean Abbott

Chapter 3. Data Preparation – Clean

Introduction

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.