A Complete Web Monitoring Maturity Model

As organizations improve their web visibility, they transition through several stages of maturity. You won’t achieve the same maturity across all four of the big questions at once. You may already have mature web analytics, for example, but only primitive insights into web performance.

Level 1: Technical Details

At the first level of maturity, monitoring is done for technical reasons. The organization hasn’t yet realized that the web application is important to its business strategy, and results are used only to ensure that systems are functioning and to detect egregious errors. If you’re at this level, you’re probably asking questions like:

  • How many hits does my site get each day? Is that number growing?

  • What percentage of HTTP GETs to the site receive an HTTP 200 OK response?

  • How slowly do my servers respond to the tests I’m running?

  • Does my website work on different web browsers?

There’s nothing fundamentally wrong with these questions, but on their own, they don’t relate to the business because they don’t relate to a particular business objective.

Consider, for example, a server that gets a thousand requests. That traffic could all be coming from one energetic user, or it could be a thousand one-page visitors who didn’t find what they wanted. Neither is good. Or consider a server that suddenly responds slowly to a test. Unless you know how many users were on your site and what they were doing, you don’t know whether the slowdown affected your business.

Level 2: Minding Your Own House

As an organization engages its customers, it first focuses on the things it can control. This leads to questions about the effectiveness of the website, such as:

  • Does the website convert visitors?

  • Is it fast and available when people need it?

  • Why do visitors come to the site?

  • What are the most common ways people use the site, and which of those lead to good outcomes (goals and conversions)?

These are the key metrics that organizations need to use to make sure the site can do what it needs to: convert visitors. They’re a great start.

Because they don’t look beyond the borders of the website to the Internet as a whole, however, organizations at this level are still inward-facing and somewhat myopic. They also fail to consider long-term metrics like the lifetime value of a visitor, loyalty, and capacity trends.

Level 3: Engaging the Internet

With your own house in order, you can now turn your attention to the rest of the Web. The main focus here is driving traffic to your site through marketing campaigns, paid search, and organic search. The site engages visitors in other ways, too, forcing you to ask new questions:

  • How are people finding the site, and what’s working best?

  • Are the other components of the Internet, such as online communities, APIs, web services, and CDNs, functioning properly?

  • What’s the Internet saying about the organization, and how is that affecting traffic rates?

  • Are different segments of the Internet showing different levels of usability? Are foreign-language browsers having to scroll down more? Are older visitors with larger font sizes seeing less of certain pages?

  • Are there trolling and spamming problems?

To deal with the flood of measurements generated by additional visitors, the organization starts to analyze data differently. For example, it uses software to automatically baseline what “normal” is, pinpoint exceptions, and predict seasonal or growth trends.

Level 4: Building Relationships

The next stage of maturity turns visits into long-term relationships. In the earlier stages of the maturity model, each visit was treated as a unique interaction. As your organization matures, these interactions are stitched together into a user record, and analytics starts to converge with CRM.

The central focus is now a visitor’s lifetime relationship with your site. You learn how often he visits the site, what triggers those visits, and what outcomes happen.

Interactions happen beyond the website. Customer loyalty cards show store visits, calls to the call center are tracked by phone number, and so on. The relationship becomes a two-way engagement in which you reach out to visitors who have opted in to mailing lists or who subscribe to RSS feeds.

A relationship-focused organization cares about longevity, loyalty, and capturing the most complete view of its visitors:

  • How often do visitors return, and what’s their lifetime value to the organization?

  • How steep is the learning curve, and how long does it take a new visitor to use 80 percent of the site’s features?

  • How do word of mouth and the community affect loyalty?

  • Which modes of interaction (web, online community, mail, phone, in-person) are most common, and what causes visitors to switch from one to another?

  • How many visits does it take before a reader comments?

  • How willing are my customers to have me communicate with them?

  • What’s my reach (ability to get to customers) and how likely are they to act on what I send them?

  • What performance or uptime expectations—or implied SLA—do my I have with my power users?

Stitching together individual visits, both offline and online, is a major challenge for any organization. If you’re at this stage, you need to handle personal information carefully so you don’t run afoul of privacy laws or put your organization at risk.

Level 5: Web Business Strategy

The final stage of maturity occurs when the business makes the Web an integral part of its product and marketing strategies. In the earlier levels of the maturity model, the Web is an “online branch office”—it often has its own profit and loss, and is run like a separate business.

But the Web touches every part of our lives. It’s on our phones and in our cars. It’s quickly becoming the dominant medium for music, video, and personal messaging. It’s how employees work remotely. It’s many people’s primary source of day-to-day knowledge. The Web’s effects reach far beyond a storefront, affecting everything from how companies develop new products to how they support existing ones.

When companies finally embrace the Web as the disruptive force it is, they stop becoming reactive and start becoming opportunistic. Instead of reacting to what web monitoring tells them, they start to wonder whether the Web makes new things possible. When you reach this stage of maturity, you’ll be asking questions like:

  • Can I develop and roll out products more quickly, and with greater accuracy?

  • Can I reduce support costs and make customers self-supporting?

  • Can I tie my services to real-world components (for example, linking my CRM portal to a GPS so salespeople can optimize their travel)?

  • Can I move into other media or gain presence elsewhere?

  • How does the Web remove barriers to entry in my industry? How does it let me erect barriers to new competitors?

  • Can I engage target markets in product design and research?

One clear sign that a company views the Web as strategic is that it starts to include Web metrics in employee performance reviews and corporate goal-setting. The Web isn’t a side business anymore: it is the business.

The Watching Websites Maturity Model

Table 4-1 shows the various levels of maturity. We’ll return to the table as we look at how to answer the four big questions.

Table 4-1. Stages of watching websites

Component

Maturity Level 1

Level 2

Level 3

Level 4

Level 5

Focus

 

Technology: Make sure things are alive.

Local site: Make sure people on my site do what I want them to.

Visitor acquisition: Make sure the Internet sends people to my site.

Systematic engagement: Make sure my relationship with my visitors and the Internet continues to grow.

Web strategy: Make sure my business is aligned with the Internet age.

Who?

 

Operations

Merchandising manager

Campaign manager/SEO

Product manager

CEO/GM

Web analytics

 

Technical details: Page view, hits. Focus on operation of the infrastructure, capacity, usage.

Conversion: How many of the visitors complete the goals I intend?

Traffic: How does the Internet learn about me, encourage visits to me?

Relationship: How often do buyers return? What’s the lifetime value? Where else do customers interact with me?

Strategy: How can I combine my brick-and-mortar and web businesses? How does the web change my company?

EUEM

Synthetic

Availability and performance: Checking to see if the site is available from multiple locations, and reporting on performance.

Transactions and components: Multistep monitoring of key processes, tests to isolate tiers of infrastructure.

Testing the Internet: Monitoring of third-party components and communities on which the application depends.

Correlation & competition: Using the relationship between load and performance; comparing yourself to the industry and public benchmarks.

Organizational planning: Using performance as the basis for procurement; uptime objectives at the executive level; quantifying outages or slow-downs financially.

RUM

Errors and performance: Counting hard errors (404, 500, TCP reset) and end user performance grouped by customer or visitor segment.

Analytics integration: Tying user experience to business outcomes within the site to maximize conversions; identifying “soft errors” in transactions.

All components, all functions, automation: Watching content from third-party sites and user actions within a page; automatically forming baselines and diagnosing exceptions.

SLA, CRM: Using RUM information as the basis for SLAs, problem resolution, and release management.

Integrated user engagement: Measuring user experience across call centers, in-person encounters, web transactions, etc., as a single entity; impact of user experience is quantifiable.

VOC

 

“Contact us” buttons and on-site feedback; emphasis on satisfaction.

Surveys within the site via opt-in invitations; emphasis on loyalty.

Engaging the public Internet (chatrooms, social sites, etc.) and analyzing key topics and discussions; emphasis on word-of-mouth and virality.

Customer collaboration in product and service design; user engagement; emphasis on lifetime value creation, giving the user a sense of ownership.

Consumer feedback tied in to corporate planning through quantitative analysis of VOC and community data; customer as a collaborator in the growth of the company.

WIA

 

Click diagrams showing “hot” areas on key pages.

Segmentation of user actions (scroll, drag, click) by outcome (purchase, abandonment, enrollment).

Segmentation by traffic source (organic search, campaign) and A/B comparison; visitor replay.

Learning curve analysis; comparison of first-time versus experienced users; automated A/B testing of usability.

Product specialization according to usability and user groups; usability as a component of employee performance.


Get Complete Web Monitoring now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.