book

APM Best Practices: Realizing Application Performance Management

Name: APM Best Practices: Realizing Application Performance Management
Author: Michael J. Sydor
ISBN: 9781430231417

by Michael J. Sydor

January 2010

Intermediate to advanced

500 pages

14h 27m

English

Apress

Read now

Unlock full access

Copyright
About the Author
Acknowledgments
Introduction
Vendor NeutralityAPM Starts HereOrganization of this BookIntroduction to Part 1: PlanningIntroduction to Part 2: ImplementationIntroduction to Part 3: Practioner's Guide
1. Getting Started with APM
1.1. The Challenge1.2. Reference Architecture1.2.1. Life Cycles1.2.1.1. The Software Development Life Cycle1.2.1.2. The Application Life Cycle1.2.1.3. The APM Life Cycle1.2.2. Organizational Maturity1.3. The Five APM Competencies1.3.1. Monitoring Architectures1.3.2. APM Architecture1.3.3. Application Characteristics1.3.4. Reporting Characteristics1.4. The First Meeting1.5. Meeting Themes1.5.1. What is APM?1.5.2. Realities of Packaged Software Solutions1.5.3. How the Monitoring Game has Changed1.5.4. Limitations of Availability Monitoring1.5.4.1. Enter Two-Tier Client Server and FCAPS1.5.4.2. The Need for a New Perspective1.5.4.3. Emphasizing Transactions in Addition to Resources1.5.5. The Impact of APM Visibility1.5.6. Addressing the Visibility Gap1.5.6.1. Getting Those New Metrics1.5.6.2. Understanding Those New Metrics1.5.6.3. Managing Those New Metrics1.5.6.4. Exploiting Those New Metrics across the Application Life Cycle1.5.7. Demonstrating Value1.5.7.1. Good-Better-Best1.5.8. Establishing an APM Project1.5.8.1. Initial Iteration1.5.8.2. Follow-on Iteration1.5.8.3. Closing Iteration1.6. Summary
I. Planning
2. Business Justification
2.1. Justification Is Not ROI2.2. Entry Points2.3. Initiative Themes2.3.1. Availability vs. Performance Monitoring2.3.2. Resolving Application Incidents and Outages2.3.3. Improving Application Software Quality2.3.4. Pre-production Readiness and Deployment2.3.5. Managing Service Level Agreements2.3.6. Enhancing the Value of the Monitoring Tool Investment2.3.7. Proactive Monitoring2.3.8. Trending and Analysis2.3.9. Single-View of Service Performance (Dashboards)2.4. Summary
3. Assessments
3.1. Overview3.1.1. Visibility3.2. Assessment Dimensions3.2.1. Application Survey3.2.1.1. Survey Dimensions3.2.1.2. Business3.2.1.3. Environment3.2.1.4. Resources3.2.1.5. Software Architecture3.2.1.6. Existing Monitoring (Optional)3.2.1.7. Avoiding Scope-Creep During Deployment Planning3.2.2. Skills and Processes3.2.3. Incidents3.2.3.1. Cookbook3.2.4. Stakeholder Interviews3.2.4.1. APM Roles3.2.4.2. Topics by Stakeholder3.2.4.3. Reporting Dimensions3.2.4.4. Cookbook3.2.4.5. Management Capabilities3.2.4.5.1. Cookbook3.2.4.6. End-to-End Visibility3.2.4.6.1. Cookbook3.2.4.7. Life Cycle Visibility3.2.4.7.1. Cookbook3.2.4.8. Monitoring Tools3.2.4.8.1. Executive Summary3.2.4.8.2. Cookbook3.2.5. Detailed Findings (Alternate Technique)3.2.6. Solution Sizing3.2.6.1. Synthetic Transactions3.2.6.2. Real Transactions3.2.6.3. Instrumentation3.2.6.4. Cookbook3.3. Summarizing Your Findings and Recommendations3.3.1.3.3.1.1. Cookbook3.4. Conclusion
4. Staffing and Responsibilities
4.1. The Staffing Question4.1.1. Who Has Responsibility for Monitoring?4.1.1.1. The APM Expert4.1.2. Roles and Responsibilities4.1.2.1. APM Administrator4.1.2.2. APM Project Manager4.1.2.3. Application Specialist4.1.2.4. APM Architect4.1.2.5. APM Specialist (Triage/Firefighter)4.1.2.6. APM Evangelist4.1.3. Staffing Strategies4.1.3.1. Adding New Personnel4.1.3.2. Product Training4.1.3.3. Repurposing Existing Personnel4.1.3.4. Best Practice Mentoring4.1.3.5. Staff Augmentation4.2. Staffing an APM Initiative4.2.1. What Staffing Is Appropriate for APM Success?4.2.2. The FTE Approach4.2.3. Evolving Your Organization's Use of APM4.2.4. Building a Scalable Monitoring Organization4.2.5. Real-World Staffing Evolution Scenarios?4.2.5.1. First APM, Accepted but Constrained Budget4.2.5.2. Significant APM, Committed but Has Gaps with Current Practice4.2.5.3. Significant Existing APM, Recommitted but Constrained Budget4.2.5.4. Significant Existing APM, Recommitted and Budgeted for APM Practice4.2.5.5. Significant Existing APM, Retrenching, Maintain Existing Footprint Only4.3. Summary
5. APM Patterns
5.1. Processes, Skills, and Competencies for APM5.2. Demonstrating Value5.3. Management Maturity5.4. Deployment Scenarios and Priorities5.4.1. Small Deployment Footprint Scenario5.4.2. Multiple Independent Initiatives Scenario5.4.3. Service Bureau for a Line of Business Scenario5.4.4. Center of Excellence for a Corporation Scenario5.5. Defining Your Services Catalog5.5.1. Why is a Services Catalog Such a Valuable Strategy?5.5.2. Assessing Role Competency5.6. The Last APM Pattern5.7. Cookbook5.8. Summary

6. The Pilot Evaluation
6.1. The Pilot Evaluation6.1.1. Participation6.1.2. Goals6.1.2.1. Connectivity6.1.2.2. Platform Suitability6.1.2.3. Application Suitability6.1.2.4. Pre-Production Visibility6.1.2.5. Production Visibility6.1.3. Criteria6.1.3.1. Compatible with your Environment6.1.3.2. Ease of Initial Installation6.1.3.3. Flexibility of Technology Configuration6.1.3.4. Assessing Overhead6.1.3.5. Assessing Deployability and Scalability6.1.3.6. Solution Certification6.1.4. Cookbook6.1.4.1. Planning Phase6.1.4.2. Scope Document6.1.4.3. Implementation6.1.4.4. Review6.2. Summary
II. Implementation
7. Deployment Strategies
7.1. Stand-Alone7.2. Phased Deployments7.2.1.7.2.1.1. Preproduction7.2.1.2. Preproduction Review7.2.1.3. Operations7.3. Realizing QA and Triage7.3.1. Understanding Preproduction7.3.2. The Problem with Preproduction7.3.2.1. Repurposing QA Equipment for Production7.3.3. Evolving QA and Triage7.3.3.1. Acceptance Criteria7.3.3.2. Triage7.3.4. Evolving Management Capabilities7.3.4.1. Reactive Management7.3.4.2. Reactive Alerting7.3.4.3. Predictive/Directed Management7.3.4.4. Proactive Management7.4. Deployment Necessities7.4.1. Kickoff Meeting7.4.2. Phased Deployment Schedule7.4.3. Preproduction Review7.5. Postproduction Review7.6. Install/Update Validation7.6.1. Call Stack Visibility7.6.2. Transaction Definition7.7. Updates7.7.1. Alternate Agent Configurations and Versions7.8. Alert Integration7.9. Summary
8. Essential Processes
8.1. Monitoring Runbook8.2. Pre-Production Practice8.3. Operations8.4. Improving Software Quality8.4.1.8.4.1.1. Define Acceptance Criteria8.4.1.2. Audit the Application8.4.1.3. Onboard the Application8.4.1.4. Supervise the Third Party Testing Process8.4.1.5. Empower the Third Party to Deliver Proactive Performance Management8.5. Summary
9. Essential Service Capabilities
9.1. Triage9.1.1.9.1.1.1. Metrics Storage Dashboard9.1.1.2. Pilot Findings9.1.1.3. Metrics Archive Triage and Presentation9.1.1.4. Baselines9.1.1.5. Characterization9.1.1.6. Cookbook9.2. Application Audit9.2.1. Why Audits Matter9.3. Pre-production Review9.4. Capacity Management of APM Metrics9.5. Solution Sizing9.6. Capacity Forecast9.6.1. Application9.6.2. Monitoring Environment9.7. Summary
III. Practitioners
10. Solution Sizing
10.1. Kick Off Meeting10.1.1. Solution Architecture10.1.1.1.10.1.1.1.1. Stand-alone10.1.1.1.2. Failover10.1.1.1.3. Federated10.1.1.2. Metrics Storage and Agent Capacities and Realities10.1.2. Metrics Storage Sizing10.1.2.1. Application Realities10.1.2.1.1. Not everything needs instrumentation monitoring10.1.2.1.2. Not all apps need detailed application monitoring.10.1.2.1.3. Some apps need KPIs.10.1.2.1.4. Not all apps function the same way.10.1.2.1.5. Most apps have different monitoring requirements across the application life cycle.10.1.2.2. Sizing Attributes10.1.3. Deployment Strategy10.1.3.1. Ideal APM Architecture10.1.4. Monitoring the Metrics Storage10.1.4.1. Why do MS Components Fail?10.1.5. Communicating Sizing Results and Recommendations10.1.6. Process Considerations10.1.7. Deployment Survey and Sizing10.1.8. Solution Certification10.1.8.1. Test Architecture10.1.8.2. Load Profiles10.1.8.3. Reporting10.1.8.4. Analysis10.2. Competency10.3. Artifacts10.4. Summary
11. Load Generation
11.1. Kick Off Meeting11.1.1. Why Simulate Load?11.1.2. Types of Testing11.1.2.1. Test Anti-patterns11.1.2.2. Test Evolution11.1.2.2.1. Maximum Load Considerations11.1.2.2.2. Defining a Test Plan11.1.2.2.3. User Population11.1.2.3. Accuracy and Reproducibility11.2. Process11.2.1. Test Plans11.2.1.1. Test Evolution—Gold Configuration (sequence of configurations)11.2.1.1.1. Test Plan Cookbook11.2.1.1.2. Gold Configuration Review11.2.1.1.3. Load Reproducibility Review11.2.1.2. Application Baseline11.2.1.2.1. Heartbeat Metrics11.2.1.3. Performance Baseline11.2.1.4. Load Reproducibility Analysis11.2.1.5. General Acceptance Criteria11.3. Competency11.4. Artifacts11.5. Summary
12. Baselines
12.1. Kick-Off Meeting12.1.1. Terms of Endearment12.1.1.1. Configuration Tuning12.1.1.2. Baseline12.1.1.3. Configuration Baseline12.1.1.4. Application Baseline12.1.1.5. Performance Baseline12.1.1.6. Capacity Planning12.1.1.7. Capacity Forecast12.1.2. The Fab Four12.1.2.1. Reporting12.1.2.1.1. Frequency12.1.2.1.2. Baseline12.1.2.1.3. HealthCheck12.1.2.1.4. Summary12.2. Process12.2.1. Collecting Baselines12.2.1.1. Configuration12.2.1.2. Application12.2.1.3. Performance12.3. Competency12.4. Summary
13. The Application Audit
13.1. Kick Off Meeting13.1.1. Compare and Contrast13.1.1.1. What to Look for13.1.1.1.1. GC/Heap13.1.1.1.2. EJB, Servlet, JSP13.1.1.1.3. Concurrency13.1.1.1.4. Stalls13.1.1.1.5. Optional13.1.1.2. Auditing Pre-Production13.1.1.3. Auditing Operationally13.1.2. Configuration Baseline13.1.2.1. Project Scope13.1.2.2. Test Plan13.1.2.3. Analysis of Results13.1.2.4. Memory Utilization13.1.2.5. CPU Utilization13.1.2.6. I/O Throughput13.1.2.7. I/O Contention13.1.2.8. Conclusions13.1.3. HealthCheck13.1.4. Dashboard and Alerts13.1.4.1.13.1.4.1.1. Managing Alerts13.1.4.2. Getting Production Baselines13.2. Process13.2.1. Scope13.2.1.1. Necessities13.2.2. Load Generation13.2.2.1. Production-Only Situation13.2.3. Transaction Definition13.2.4. Acceptance Criteria13.2.5. Reporting13.3. Competency13.4. Summary
14. Triage with Single Metrics
14.1. Triage with Single Metrics14.1.1.14.1.1.1. The Program14.2. Kick Off Meeting14.2.1. Motivation14.2.2. Why Triage is Difficult to Master14.2.2.1. Triage with Single Metrics14.2.2.1.1. Navigation Among Single Metrics14.2.2.1.2. Organization via Metrics Collections14.2.2.1.3. Presentation via Dashboards14.2.2.1.4. Presentation via Reports14.2.2.2. Metric Categories14.2.2.2.1. Memory Profile14.2.2.2.2. Response Times14.2.2.2.3. Concurrency14.2.2.2.4. Stalls14.2.2.2.5. SQL14.3. Process14.3.1. Scope14.3.2. Triage with Single Metrics14.3.2.1. Single Incident14.3.2.2. Performance Test14.3.2.2.1. Initial14.3.2.2.2. Baseline Report Available14.3.2.3. HealthCheck14.3.2.3.1. QA HealthCheck14.3.2.3.2. Production HealthCheck14.4. Competency14.5. Artifacts14.5.1. Triage scope Template14.6. Summary
15. Triage with Baselines
15.1. Kick Off Meeting15.1.1. Motivation15.1.2. Triage with Baselines15.1.2.1. Reporting Conventions15.1.2.2. Achieving Consistent Results15.1.2.3. Component Analysis15.1.2.4. Initial Automation15.2. Process15.2.1. Triage with Baselines15.3. Summary
16. Triage with Trends
16.1. Kick Off Meeting16.1.1. Motivation16.1.1.1. Types of Trends: Correlation16.1.1.2. Trending Scenario16.1.1.3. Preparing for Triage with Trends16.1.1.3.1. Data Integration Strategies16.1.1.3.2. Extending Visibility16.1.1.3.3. Reporting Integration16.1.1.4. The Future of Monitoring Integration16.1.1.4.1. Integration Strategies16.1.1.4.2. APM-Specific Applications for CMDB16.1.1.5. Root-cause Analysis16.1.1.6. Implementing the Fix16.1.1.7. Service Level Management16.1.1.7.1. Defining the SLA16.2. Process16.2.1. Triage with Trends16.3. Competency16.4. Artifacts16.5. Summary
17. Firefighting and Critical Situations
17.1. Kick Off Meeting17.1.1. Firefighter, Smoke-jumpers, Triage17.1.2. Before You Jump17.1.2.1. Do You Have the Right Tools?17.1.2.2. Do You Have Potential Data?17.1.2.3. Do You Have an Appropriate Environment?17.1.2.4. Have You Set Appropriate Expectations?17.1.3. Why Do Applications Fail?17.1.4. What May Firefighting Achieve?17.1.5. Resistance Sounds Like...17.1.6. Success Sounds Like...17.1.7. Firefighting is Temporary17.2. Process17.2.1. Firefighting Process Documentation17.2.1.1. Scope document17.2.1.2. Scope Template17.2.1.3. Rules of Engagement (internal to the firefight organization)17.2.2. Forensic Analysis17.2.3. Establishing the Firefighting Discipline17.2.3.1. Get Visibility17.2.3.1.1. Checklist17.2.3.2. Audit the Production Experience17.2.3.3. Audit the QA Experience with Recommendations17.3. Competency17.3.1. Rapid Deployment17.3.2. Analysis and Presentation17.3.3. Application Audit17.3.4. Performance Tuning17.4. Summary

Content preview from APM Best Practices: Realizing Application Performance Management

Chapter 1. Getting Started with APM

The second mouse gets the cheese.

The Challenge

Software systems, the applications and strategies on which your business depend today have become extraordinarily complex. For many organizations, the wide array of applications and tools to develop this software results in application architectures that can overwhelm the capabilities of many organizations to manage them.

Not so long ago, how you implemented your software was hidden deep within your organization. Typically, your interactions consisted of exchanges of data representing inventory, orders, debits, and credits that were transferred across private networks. These interactions involved a limited number of other corporations, loosely called trading partners ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Application Performance Management (APM) in the Digital Enterprise

Publisher Resources

ISBN: 9781430231417Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

APM Best Practices: Realizing Application Performance Management

by Michael J. Sydor

Chapter 1. Getting Started with APM

The Challenge

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.