book

Evaluation Theory, Models, and Applications, 2nd Edition

by Daniel L. Stufflebeam, Chris L. S. Coryn

October 2014

Intermediate to advanced

800 pages

29h 19m

English

Jossey-Bass

Read now

Unlock full access

Title Page
Copyright
List of Figures, Tables, and Exhibits
FiguresTablesExhibits
Dedication
Preface
Acknowledgments
The Authors
Introduction
Changes to the First EditionIntended AudienceOverview of the Book's ContentsStudy SuggestionsSummary
Part One: Fundamentals of Evaluation
Chapter 1: Overview of the Evaluation Field
What Are Appropriate Objects of Evaluations and Related Subdisciplines of Evaluation?Are Evaluations Enough to Control Quality, Guide Improvement, and Protect Consumers?Evaluation as a Profession and Its Relationship to Other ProfessionsWhat Is Evaluation?How Good Is Good Enough? How Bad Is Intolerable? How Are These Questions Addressed?What Are Performance Standards? How Should They Be Applied?Why Is It Appropriate to Consider Multiple Values?Should Evaluations Be Comparative, Noncomparative, or Both?How Should Evaluations Be Used?Why Is It Important to Distinguish Between Informal Evaluation and Formal Evaluation?How Do Service Organizations Meet Requirements for Public Accountability?What Are the Methods of Formal Evaluation?What Is the Evaluation Profession, and How Strong Is It?What Are the Main Historical Milestones in the Evaluation Field's Development?SummaryGroup ExercisesNotesSuggested Supplemental Readings

Chapter 2: Evaluation Theory
General Features of Evaluation TheoriesTheory's Role in Developing the Program Evaluation FieldFunctional and Pragmatic Bases of Extant Program Evaluation TheoryA Word About Research Related to Program Evaluation TheoryProgram Evaluation Theory DefinedCriteria for Judging Program Evaluation TheoriesTheory Development as a Creative Process Subject to Review and Critique by UsersStatus of Theory Development in the Program Evaluation FieldImportance and Difficulties of Considering Context in Theories of Program EvaluationNeed for Multiple Theories of Program EvaluationHypotheses for Research on Program EvaluationPotential Utility of Grounded TheoriesPotential Utility of Metaevaluations in Developing Theories of Program EvaluationProgram Evaluation Standards and Theory DevelopmentSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 3: Standards for Program Evaluations
The Need for Evaluation StandardsBackground of Standards for Program EvaluationsJoint Committee Program Evaluation StandardsAmerican Evaluation Association Guiding Principles for EvaluatorsGovernment Auditing StandardsUsing Evaluation StandardsSummaryGroup ExercisesNotesSuggested Supplemental Readings
Part Two: An Evaluation of Evaluation Approaches and Models
Chapter 4: Background for Assessing Evaluation Approaches
Evaluation ApproachesImportance of Studying Alternative Evaluation ApproachesThe Nature of Program EvaluationPrevious Classifications of Alternative Evaluation ApproachesCaveatsSummaryGroup ExerciseSuggested Supplemental Readings
Chapter 5: Pseudoevaluations
Background and IntroductionApproach 1: Public Relations StudiesApproach 2: Politically Controlled StudiesApproach 3: Pandering EvaluationsApproach 4: Evaluation by PretextApproach 5: Empowerment Under the Guise of EvaluationApproach 6: Customer Feedback EvaluationSummaryGroup ExercisesNotesSuggested Supplemental Readings
Chapter 6: Quasi-Evaluation Studies
Quasi-Evaluation Approaches DefinedFunctions of Quasi-Evaluation ApproachesGeneral Strengths and Weaknesses of Quasi-Evaluation ApproachesApproach 7: Objectives-Based StudiesApproach 8: The Success Case MethodApproach 9: Outcome Evaluation as Value-Added AssessmentApproach 10: Experimental and Quasi-Experimental StudiesApproach 11: Cost StudiesApproach 12: Connoisseurship and CriticismApproach 13: Theory-Based EvaluationApproach 14: Meta-AnalysisSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 7: Improvement- and Accountability-Oriented Evaluation Approaches
Improvement- and Accountability-Oriented Evaluation DefinedFunctions of Improvement- and Accountability-Oriented ApproachesGeneral Strengths and Weaknesses of Decision- and Accountability-Oriented ApproachesApproach 15: Decision- and Accountability-Oriented StudiesApproach 16: Consumer-Oriented StudiesApproach 17: Accreditation and CertificationSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 8: Social Agenda and Advocacy Evaluation Approaches
Overview of Social Agenda and Advocacy ApproachesApproach 18: Responsive or Stakeholder-Centered EvaluationApproach 19: Constructivist EvaluationApproach 20: Deliberative Democratic EvaluationApproach 21: Transformative EvaluationSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 9: Eclectic Evaluation Approaches
Overview of Eclectic ApproachesApproach 22: Utilization-Focused EvaluationApproach 23: Participatory EvaluationSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 10: Best Approaches for Twenty-First-Century Evaluations
Selection of Approaches for AnalysisMethodology for Analyzing and Evaluating the Nine ApproachesOur Qualifications as RatersConflicts of Interest Pertaining to the RatingsStandards for Judging Evaluation ApproachesComparison of 2007 and 2014 RatingsIssues Related to the 2011 Program Evaluation StandardsOverall ObservationsThe Bottom LineSummaryGroup ExercisesNotesSuggested Supplemental Readings
Part Three: Explication of Selected Evaluation Approaches
Chapter 11: Experimental and Quasi-Experimental Design Evaluations
Chapter OverviewBasic Requirements of Sound ExperimentsProspective Versus Retrospective Studies of CauseUses of Experimental DesignRandomized Controlled Experiments in ContextSuchman and the Scientific Approach to EvaluationContemporary Concepts Associated with the Experimental and Quasi-Experimental Design Approach to EvaluationExemplars of Large-Scale Experimental and Quasi-Experimental Design EvaluationsGuidelines for Designing ExperimentsQuasi-Experimental DesignsSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 12: Case Study Evaluations
Overview of the ChapterOverview of the Case Study ApproachCase Study Research: The Views of Robert StakeCase Study Research: The Views of Robert YinParticular Case Study Information Collection MethodsSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 13: Daniel Stufflebeam's CIPP Model for Evaluation: An Improvement- and Accountability-Oriented Approach
Overview of the ChapterCIPP Model in ContextOverview of the CIPP CategoriesFormative and Summative Uses of Context, Input, Process, and Product EvaluationsPhilosophy and Code of Ethics Underlying the CIPP ModelThe Model's Values ComponentUsing the CIPP Framework to Define Evaluation QuestionsDelineation of the CIPP Categories and Relevant ProceduresUse of the CIPP Model as a Systems Strategy for ImprovementSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 14: Michael Scriven's Consumer-Oriented Approach to Evaluation
Overview of Scriven's Contributions to EvaluationScriven's BackgroundScriven's Basic Orientation to EvaluationScriven's Definition of EvaluationCritique of Other PersuasionsFormative and Summative EvaluationAmateur Versus Professional EvaluationIntrinsic and Payoff EvaluationGoal-Free EvaluationNeeds AssessmentScoring, Ranking, Grading, and ApportioningChecklistsKey Evaluation ChecklistThe Final SynthesisMetaevaluationEvaluation IdeologiesAvenues to Causal InferenceProduct EvaluationProfessionalization of EvaluationScriven's Look to Evaluation's FutureSummaryGroup ExercisesNotesSuggested Supplemental Readings
Chapter 15: Robert Stake's Responsive or Stakeholder-Centered Evaluation Approach
Stake's Professional BackgroundFactors Influencing Stake's Development of Evaluation TheoryStake's 1967 “Countenance of Educational Evaluation” ArticleResponsive Evaluation ApproachSubstantive Structure of Responsive EvaluationFunctional Structure of Responsive EvaluationAn Application of Responsive EvaluationStake's Recent Rethinking of Responsive EvaluationSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 16: Michael Patton's Utilization-Focused Evaluation
Adherents of Utilization-Focused EvaluationSome General Aspects of Patton's Utilization-Focused EvaluationIntended Users of Utilization-Focused EvaluationFocusing a Utilization-Focused EvaluationThe Personal Factor as Vital to an Evaluation's SuccessThe Evaluator's RolesUtilization-Focused Evaluation and Values and JudgmentsEmploying Active-Reactive-Adaptive Processes to Negotiate with UsersPatton's Eclectic ApproachPlanning Utilization-Focused EvaluationsCollecting and Analyzing Information and Reporting FindingsSummary of Premises of Utilization-Focused EvaluationStrengths of the Utilization-Focused Evaluation ApproachLimitations of the Utilization-Focused Evaluation ApproachSummaryGroup ExercisesNoteSuggested Supplemental Readings
Part Four: Evaluation Tasks, Procedures, and Tools
Chapter 17: Identifying and Assessing Evaluation Opportunities
Sources of Evaluation OpportunitiesBidders' ConferencesSummaryGroup ExercisesSuggested Supplemental Reading
Chapter 18: First Steps in Addressing Evaluation Opportunities
Developing the Evaluation TeamDeveloping Thorough Familiarity with the Need for the EvaluationStipulating Standards for Guiding and Assessing the EvaluationEstablishing Institutional Support for the Projected EvaluationDeveloping the Evaluation Proposal's AppendixPlanning for a Stakeholder Review PanelSummaryGroup ExerciseSuggested Supplemental Readings
Chapter 19: Designing Evaluations
A Design Used for Evaluating the Performance Review System of a Military OrganizationGeneric Checklist for Designing EvaluationsSummarySuggested Supplemental Readings
Chapter 20: Budgeting Evaluations
Ethical Imperatives in Budgeting EvaluationsFixed-Price Budget for Evaluating a Personnel Evaluation SystemOther Types of Evaluation BudgetsGeneric Checklist for Developing Evaluation BudgetsSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 21: Contracting Evaluations
Definitions of Evaluation Contracts and Memorandums of AgreementRationale for Evaluation ContractingAddressing Organizational Contracting RequirementsNegotiating Evaluation AgreementsEvaluation Contracting ChecklistSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 22: Collecting Evaluative Information
Key Standards for Information CollectionAn Information Collection FrameworkUseful Methods for Collecting InformationSummarySuggested Supplemental Readings
Chapter 23: Analyzing and Synthesizing Information
General Orientation to Analyzing and Synthesizing InformationPrinciples for Analyzing and Synthesizing InformationAnalysis of Quantitative InformationAnalysis of Qualitative InformationJustified Conclusions and DecisionsSummaryGroup ExercisesSuggested Supplemental Readings
Chapter 24: Communicating Evaluation Findings
Review of Pertinent Analysis and Advice from Previous ChaptersComplex Needs and Challenges in Reporting Evaluation FindingsEstablishing Conditions to Foster Use of FindingsProviding Interim Evaluative FeedbackPreparing and Delivering the Final ReportProviding Follow-Up Support to Enhance an Evaluation's ImpactSummaryGroup ExercisesSuggested Supplemental Readings
Part Five: Metaevaluation and Institutionalizing and Mainstreaming Evaluation
Chapter 25: Metaevaluation: Evaluating Evaluations
Rationale for MetaevaluationEvaluator and Client Responsibilities in Regard to MetaevaluationFormative and Summative MetaevaluationsA Conceptual and Operational Definition of MetaevaluationAn Instructive Metaevaluation CaseMetaevaluation TasksMetaevaluation Arrangements and ProceduresComparative MetaevaluationsChecklists for Use in MetaevaluationsThe Role of Context and Resource ConstraintsSummaryGroup ExercisesNoteSuggested Supplemental Readings
Chapter 26: Institutionalizing and Mainstreaming Evaluation
Review of this Book's ThemesOverview of the Remainder of the ChapterRationale and Key Principles for Institutionalizing and Mainstreaming EvaluationEarly Efforts to Help Organizations Institutionalize EvaluationRecent Advances of Use in Institutionalizing and Mainstreaming EvaluationChecklist for Use in Institutionalizing and Mainstreaming EvaluationSummaryGroup ExercisesSuggested Supplemental Readings
Glossary
References
Index
End User License Agreement

Content preview from Evaluation Theory, Models, and Applications, 2nd Edition

Chapter 5Pseudoevaluations

Background and Introduction

Because this book is aimed at examining and explaining the state of the art in evaluation, it is necessary to discuss bad and questionable practices as well as best efforts. Evaluators and their clients are sometimes tempted to shade, selectively release, overgeneralize, or even falsify findings. In addition, evaluators might falsely characterize constructive efforts—such as providing evaluation training or developing an organization's evaluation ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Bayesian Data Analysis, Third Edition, 3rd Edition

Publisher Resources

ISBN: 9781118870228Other

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Evaluation Theory, Models, and Applications, 2nd Edition

by Daniel L. Stufflebeam, Chris L. S. Coryn

Chapter 5Pseudoevaluations

Background and Introduction

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.