book

CMMI® Assessments: Motivating Positive Change

by Marilyn Bush, Donna Dunaway

February 2005

Beginner

432 pages

8h 19m

English

Addison-Wesley Professional

Read now

Unlock full access

Copyright
Dedication
Foreword
How It All StartedEarly Software ExperiencesAssessment IssuesThe SEI and the CMMCMM DevelopmentWhat Next?
Preface
Intended Audience for the BookAssessments Are Part of the Larger Subject of Process ImprovementSoftware Versus Systems and Hardware Process Improvement AssessmentsCMM/CBA IPI Versus CMMI/SCAMPIStaged Versus Continuous AssessmentsAnecdotal HistoriesA Roadmap Through the Book
Acknowledgments
About the Authors
1. Why do Assessments?
1.1. What Assessments Do1.2. The Four Principal Functions of Assessments1.2.1. Function 1: Assessments Serve as Analytical Tools1.2.2. Function 2: Assessments Function as Fulcrums of Positive Change1.2.3. Function 3: Assessments Transform Organizations by the Way They Work1.2.4. Function 4: Assessments Educate Organizations in Worldwide Best Practices1.3. The Analytical Function of Assessments1.3.1. The Importance of Reference Models1.3.2. Assessments Stabilize Process and Prioritize Change1.4. Assessments Function as Fulcrums of Positive Change1.4.1. Assessments Effect Change by Involving and Motivating Organizations in Efforts of Self-Analysis1.4.2. Assessments Effect Change Because They Help the Workers in an Organization Understand That Processes, Not People, Need to Be Fixed1.4.3. Assessments Effect Change Because They Provide a Voice for Change Agents1.4.4. Assessments Effect Change Because They Foster Follow-On Activities1.5. Assessments Transform Organizations by the Way They Work1.5.1. Assessments Transform Organizations by Getting Different People to See the Same Things the Same Way1.5.2. Assessments Transform Organizations by Helping Senior Management’s Efforts at Unification1.5.3. Assessments Transform Organizations by Softening a Culture of Blame, Permitting Staff the Freedom to Think About What Goes Wrong and How to Correct It1.5.4. Assessments Transform Organizations by Encouraging People to Think Across Boundaries1.5.5. Assessments Transform Organizations by Consolidating a Party of Improvement1.5.6. Assessments Transform Organizations by Helping to Institutionalize Rigorous Analysis1.6. Assessments Educate as They Analyze, Motivate, and Transform1.7. Why Gaming the Results of an Assessment Doesn’t Help (Though Many Try)1.8. Can Assessments Really Change an Organization? A Preview of an Extended Case History to Be Found in Chapter 121.8.1. Analysis1.8.2. Initiation of Positive Change1.8.3. Positive Transformation1.8.4. Education1.8.5. A Continuous Program of Assessment and Improvement1.9. Bottom-Line Profit and Cost Numbers: Assessments Pay
2. A Brief History of Process Improvement Methodologies and Assessment Methods
2.1. The Beginnings of Modern Software Assessment Methodology2.1.1. The Creation of the SEI2.1.2. Assessments Versus Evaluations2.1.2.1. The Inconsistency of Early SPA Results and Its Upshot2.2. The SEI Capability Maturity Model2.2.1. Capability Maturity Levels2.2.2. The Structure and Philosophy of the CMM2.2.3. Benefits from Succeeding Maturity Levels2.2.3.1. Benefits from Maturity Level 22.2.3.2. Benefits from Maturity Level 32.2.3.3. Benefits from Maturity Level 42.2.3.4. Benefits from Maturity Level 52.2.4. The CMM Assessment Method: CBA IPI2.3. The Three Principal Advantages of a Modern Software Assessment over Traditional Manufacturing-Based Audit Procedures(1) A Focus on the Reliability of the Organization as a Whole(2) Encouraging Creativity and Initiative by Emphasizing Goals Instead of Means(3) An Action-Oriented Approach2.4. A Second Path Toward Software Process Improvement Assessments: The History of ISO-9000-3, Bootstrap, SPICE, and the CMMI2.4.1. ISO 90002.4.2. Bootstrap2.4.3. SPICE2.4.4. From SPICE to the CMMI: “Continuous” Versus “Staged” Assessments2.5. The CMMI: An Enlarged Structure and Scope2.6. A Hybrid Assessment Approach: The CMMI SCAMPI2.6.1. Differences Between the “Staged” SCAMPI and the CBA IPI Method2.6.2. More Differences Between the SCAMPI and CBA IPI Methods2.7. Informal or Reduced Assessments: Class B and Class C Assessments
3. Assessments: An Executive Overview
3.1. What Are a Senior Executive’s Responsibilities?3.1.1. Choosing an Assessment Methodology and Reference Model3.1.2. Selecting a Lead Assessor3.1.3. Establishing (with the Lead Assessor) the Business Goals and the Scope of the Assessment3.1.3.1. Business Goals3.1.3.2. Organizational Scope3.1.3.3. Reference Model Scope3.1.4. Appointing an Assessment Sponsor from Senior Management3.1.4.1. The Responsibilities of the Assessment Sponsor3.1.5. Establishing Appropriate Organizational Understanding3.2. What Are the Phases of an Assessment?3.2.1. Phase One of an Assessment: Planning and Preparation Activities3.2.1.1. Choosing a Time3.2.1.2. Appointing an Organization Site CoordinatorThe Responsibilities of the Organization Site Coordinator3.2.1.3. The Assessment Plan3.2.1.4. Creating an Assessment Team3.2.1.5. Defining the Final Assessment Products3.2.1.6. Selecting Projects to Be Assessed3.2.1.7. Selecting People to Be Interviewed3.2.1.8. Distribution of Questionnaires3.2.1.9. Assessment Team Training3.2.1.10. The Assessment Team’s Last Pre-Onsite Activities3.2.2. Phase Two of the Assessment: Onsite Activities3.2.2.1. The Kick-Off Meeting and Other Presentations3.2.2.2. Collecting and Managing Data3.2.2.2.1. Interviews3.2.2.2.2. Consolidating Data from Interviews and Other Sources3.2.3. Phase Three: Summing Up and Presenting the Results3.2.3.1. Consolidating Draft Findings3.2.3.2. Draft Findings Meetings3.2.3.3. Final Findings Consolidation and the Establishment of Ratings, Usually Including a Capability Maturity Level Rating3.2.3.4. Presenting Final Findings Informally to the Assessment Sponsor (Optional)3.2.3.5. The Final Findings Presentation3.2.3.6. Post-Final Findings Meeting Executive Session (Optional)3.2.3.7. Assessment Team Wrap-Up3.3. Cost: How Much Time and Effort Does an Assessment Require?3.3.1. Time and Effort3.3.1.1. Resources and Time Estimates for SCAMPI3.3.2. Summary
4. Planning and Preparing for an Assessment, Part 1: Senior Management Responsibilities
4.1. Selecting a Lead Assessor4.1.1. Should a Lead Assessor Be Selected from Outside or Inside the Organization?4.2. Determining the Business Goals and the Organizational and Reference Model Scope of the Assessment4.2.1. Organizational Scope (Choosing a Division or Cross-Section of the Company to Be Assessed)4.2.1.1. The Special Problem of Matrix Organizations4.2.1.2. The Special Problem of Small Organizations4.2.2. Reference Model Scope: Which Maturity Levels Should Be Assessed?4.3. Choosing an Assessment Sponsor4.4. Establishing Appropriate Organizational Understanding
5. Planning and Preparing for an Assessment, Part 2: Choosing a Time. Formulating an Assessment Plan. Appointing an Organization Site Coordinator and Organizing Logistics
5.1. Choosing a Time for the Assessment5.2. The Assessment Plan5.2.1. How Far in Advance to Begin Formulating the Assessment Plan5.2.2. How (and by Whom) the Assessment Plan Is Written5.2.3. General Features5.2.3.1. The Assessment Plan: Identifying Confidence Factors and Risks in the Coming Assessment5.2.4. While the Details of the Assessment Plan Are Being Finalized: A List of Interim Tasks5.2.5. A Full-Scale Template of a Working Assessment Plan5.2.6. The Assessment Plan: Managing Resources and Estimating Costs5.3. Managing Logistics: Appointing an Organization Site Coordinator5.3.1. The Logistical Responsibilities of the Organization Site Coordinator and Associated Staff5.3.2. A Sample Logistics Checklist5.4. Assessment Readiness: When Is an Organization Ready for an Assessment?

6. Planning and Preparing for an Assessment, Part 3: Creating an Assessment Team. Selecting Projects to Be Assessed. Selecting People to Be Interviewed. Defining the Final Assessment Products. Distributing Questionnaires
6.1. Selecting the Assessment Team6.1.1. How Big Should the Team Be?6.1.2. Insiders Versus Outsiders6.1.3. Official CBA IPI and SCAMPI Team Member Selection Guidelines6.1.4. Selecting Assessment Team Members: Sensitive Areas6.2. Selecting Projects to Be Assessed6.2.1. Project Selection Matrix Checklist (See 5.2.5)6.3. Selecting People to Be Interviewed6.3.1. Selecting People to Be Interviewed: Senior and Middle Management6.3.2. Selecting People to Be Interviewed: Project Managers6.3.3. Selecting People to Be Interviewed: Technical Groups6.3.4. Selecting People to Be Interviewed: A Checklist Matrix (See 5.2.5)6.4. Defining Final Assessment Products6.5. Distributing Questionnaires6.5.1. Specimen Section of the CBA IPI Maturity Questionnaire (Section for the Requirements Management KPA)
7. Planning and Preparing for an Assessment, Part 4: Assessment Team Training and Post-Training Activities
7.1. Assessment Team Training7.1.1. The Lead Assessor’s Team Training and Leadership Responsibilities7.1.2. Training the Assessment Team in the Reference Model7.1.3. Training the Team in the CBA IPI and SCAMPI Assessment Methodologies7.1.4. Topics Covered During an Assessment Team Training Course7.1.5. Assessment Team Training: Team BuildingTeam Partnership: Sensitive Areas7.2. The Assessment Team’s Pre-Onsite Organization and Activities7.2.1. Managing and Facilitating Responsibilities for the Tasks Ahead7.2.2. The Assessment Team Examines Questionnaire Responses7.2.3. The Assessment Team Begins to Review Documents7.2.4. The Assessment Team Elaborates the Assessment Plan7.2.5. The Assessment Team Begins to Develop Interview Questions7.3. Preparing Organization Participants for What Is to Come7.3.1. Pre-Onsite Assessment Participants Briefing7.3.2. A Sample Assessment Participants Briefing
8. Onsite Activities, Part 1: The Kick-Off Meeting and Other Presentations. Collecting and Managing Documents Throughout the Assessment. Problems Associated with Immature Organizations
8.1. The Kick-Off Meeting and Other Presentations8.1.1. The Opening MeetingOpening Meeting Features for Organizations with Little Assessment Experience8.1.2. Other Presentations8.2. Collecting and Managing Documents Throughout the Assessment8.2.1. Reviewing Organization-Level Documents8.2.2. Reviewing Project-Level Documents8.2.3. Reviewing Implementation-Level Documents8.2.4. How the Assessment Team Prioritizes Documents for Review8.2.5. Reviewing Documents for General Context8.2.6. Library Management8.3. Problems Associated with an Immature Organization’s Desire to “Do Well” on an Assessment8.3.1. The Abuse of Mock Interviews8.3.2. Violating Confidentiality8.3.3. Intimidation8.3.4. Deliberate Obstruction8.3.5. How an Experienced Lead Assessor Should Handle Sensitive Situations
9. Onsite Activities, Part 2: Interviewing
9.1. Interviewing: An Overview9.2. Interviewing Dynamics9.3. Assessment Team Roles During the Interview9.4. The Stages of an Interview9.4.1. Opening the Interview9.4.2. Phases of the Interview9.4.3. Typical Sequences of Questions in an Interview9.4.4. Typical Time Allotments for Interview Questions9.4.5. Looking for Specific Information: Kinds of Interview Questions9.4.6. Direct or Open-Ended Questions in an Interview9.4.7. Additional Sample Questions9.4.8. More Details: Sample Technical Questions Concerning the Area of Software Project Planning (Similar Questions Must Be Formulated for Every Technical Area)9.4.9. Judgmental Statements Are to Be Avoided9.4.10. Closing the Interview9.5. Note Taking9.6. Different Interviews for Different Jobholders9.6.1. Project Manager Interviews9.6.1.1. Unexpected and/or Sensitive Situations9.6.1.2. Project Manager Seating Arrangement: A Sample Configuration9.6.2. Group Interviews9.6.2.1. Sensitive Situations in Group Interviews9.6.2.2. Two Variations of a Group Interview Seating Configuration9.6.3. Middle Manager Interviews9.6.4. Senior Management Interviews9.6.5. SCAMPI Interviews
10. Onsite Activities, Part 3: The Day-to-Day Consolidation of Data
10.1. Consolidating Data: An Overview10.2. Team Members Take Notes and Prepare to Construct “Observations” About Questionnaires, Documentation Reviews, Presentations, and Interviews10.3. Transforming Notes into Observations10.3.1. General Rules10.3.2. Two Common Problems: Under- and Over-Emphasizing the Relevance of Information10.3.3. Transferring Notes into Observations: Examples10.3.3.1. An Example of Different Kinds of Note Taking10.3.3.2. Observation Statements Based on Notes10.3.4. Observations: Suitable Recording Media10.4. How Consolidation Produces Day-to-Day Alterations in the Assessment Plan10.5. Consolidation Is a Consensus Process10.6. Warning: Consensus Must Not Be Deferred Until the Final Stages of an Assessment10.7. A Lurking Disaster to Consensus: Misunderstanding the Model10.8. The Special Requirements of the SCAMPI Consolidation Approach10.8.1. Verify Objective Evidence
11. The Final Stages of an Onsite Assessment: Summing Up and Presenting Results
11.1. Consolidating Draft Findings11.1.1. Draft Findings Presentation—A Sample Template11.2. Draft Findings Meetings: An Overview11.2.1. There Can Be Multiple Draft Findings Meetings11.2.2. Practitioner Draft Findings Meetings11.2.3. Management Draft Findings Meetings11.3. The Team’s Final Consolidation: Ratings, Including the Maturity Level Rating11.3.1. Final Consolidation11.3.2. Rating: The Process11.3.2.1. Prerequisites for Rating11.3.2.2. Final Rating Process11.3.2.3. Rating Goals11.3.2.4. Rating of Process Areas/Key Process Areas11.3.2.5. Maturity Level Ratings11.3.2.6. An Example of How Initial Questions Lead to Rating Determinations as Traced from a Technical Interview Through Final Findings11.3.3. Ratings: The Process Is Not Always Smooth11.3.3.1. Coming to Consensus About Rating KPAs/PAs11.3.3.2. How Lead Assessors can Negotiate Instances of Intransigence as Ratings Are Consolidated and Assigned11.4. CMMI Continuous Model11.4.1. Rating11.4.2. Determining the Capability Profile11.5. The Preparation of Final Findings11.5.1. Final Findings Presentation—A Sample Template11.5.2. Examples from Final Findings Presentations11.6. Presenting Final Findings Informally to Senior Management (Optional)11.7. The Final Findings Presentation11.8. Post-Final Findings Executive Session (Optional)11.9. Assessment Wrap-Up11.9.1. Organization Wrap-Up11.9.2. Assessment Data Sent to the SEI Appraisal Database11.9.3. Feedback Relating to the Effectiveness of the Assessment Method11.9.4. Confidence Report Template for CBA IPI11.9.5. Assessment Acknowledgement Letter—Template
12. How to Use the Results of an Assessment Productively
12.1. Introduction: After an Assessment12.1.1. What Happens the First Day After an Assessment12.2. Who? (Who Drives a Disciplined Post-Assessment Plan? Who Makes It Work?)12.2.1. Senior Management: The Push from Above12.2.2. Two Time-Honored Mistakes12.2.2.1. Mistake 1: Appointing a Process Improvement Manager Who Has No Real Authority12.2.2.2. Mistake 2: Hiring a Consultant to Make Everything Better12.3. When Should Post-Assessment Planning Begin? How Ambitious Should It Be?12.3.1. What Should the Duration Be of a Post-Assessment Plan?Medium TermLong Term12.4. What Does a Post-Assessment Improvement Plan Look Like? How Should It Unfold? In What Spirit Should It Be Undertaken?12.4.1. The Principles of Building a Post-Assessment Process Improvement Action Plan12.4.2. The Contents of a Post-Assessment Process Improvement Action Plan: An Overview12.4.3. A Template of a Simple Plan to Identify and Track the Execution of Improvement Goals12.4.4. The Components of a Thorough Process Improvement Plan: A Template Designed for a Mature Organization12.4.5. A Common Temptation: Trying to Run Before You Walk12.4.6. Why Maturity Levels Can’t Be Skipped12.4.7. Planning for Reiterated Assessments12.5. After the Plan: Managing the Introduction of Improved Processes12.5.1. The Role of Senior Management in Implementing Improved Processes12.5.1.1. Learning to Ask the Right Questions12.5.1.2. Extending the Authority of Senior Management Without Diluting It12.5.2. How to Create a Workable Ladder of Authority: An Example12.5.3. The Role of Process Groups in Immature Organizations12.6. Creating, Tracking, and Implementing a Post-Assessment Plan for Process Improvement12.6.1. Background12.6.2. A Summary of the First Assessment’s Findings12.6.2.1. Organization Z: Final Findings from the First Assessment12.6.2.2. The Assessment Findings: A Summary12.6.3. Organization Z’s Post-Assessment Efforts and the Difference They Made One and Two Years Afterwards12.6.4. Two Years Later: Organization Z’s Second CBA IPI Assessment12.6.4.1. Organization Z’s Second Assessment: Summary of Achievements12.6.4.2. Organization Z’s Second Assessment: Remaining Issues12.6.4.3. Organization Z Second Assessment Findings12.6.4.4. A Sample of Organization Z’s Proposals After a Second Assessment and What Difference They Made One and Two Years Later12.6.5. Four Years Later: Organization Z’s Third CBA IPI Assessment, Undertaken Two Years After the Second Assessment and Four Years After the First Assessment12.6.5.1. Organization Z’s Third Assessment: Summary of Achievements12.6.5.2. Organization Z’s Third Assessment Findings12.6.6. The Payoffs for a Level 5 Organization12.6.6.1. The Bottom Line: Productivity and Performance Improvements in Organization Z12.6.6.2. The Benefits of Reiterated Assessment in Company Z: Analysis, Positive Change, Transformation, and EducationAnalysisInitiation of Positive ChangePositive TransformationEducationA Continuous Program of Assessment and Improvement
References

Overview

Use Assessments to Drive Process Improvements in Software, Systems, Human Resources, and Beyond

Pioneered by the world's leading innovators in software engineering, assessments have evolved into a remarkably powerful tool for process improvement in areas ranging from systems to services, hardware to human resources.

Unlike audits, assessments are conducted from the inside. When handled correctly, assessments can help transform technical and managerial culture, dramatically improving both quality and profitability. In CMMI Assessments: Motivating Positive Change, two of the field's most respected leaders show exactly how to use them for maximum business advantage. Writing for executives, managers, technical professionals, and assessors themselves, Marilyn Bush and Donna Dunaway illuminate every phase of the assessment process—from planning through post-assessment follow-up.

The authors begin with an expert overview of what assessments entail, when they make sense, how to set achievable goals for them, and how to lead them to success. Next, they "drill down" into each stage of the process, presenting step-by-step instructions and defining the roles and responsibilities of every participant. Coverage includes creating and training assessment teams; identifying assessment products; consolidating interview data and other onsite activities; presenting results; and using those results productively.

Drawing on their unsurpassed experience leading assessments and mentoring assessors, they offer deep insights into the real-world challenges and obstacles you'll face—and proven solutions. They also present an extended case study showing how a real software development organization drove continuous improvement through four years of iterative assessments—moving from CMM Level 2 to elite Level 5 status, and driving dramatic business benefits along the way.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 0321179358Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills