Democratizing Analytics

Chapter 1. Data and Analytics Challenges in Modern Organizations

Accessing and analyzing data presents difficulties for every organization. In most organizations, data follows a path from creation to insight that is both delayed and restrictive. The flow is usually something similar to Figure 1-1.

Although businesses may follow different processes, common friction points exist. For example, any juncture where data is collected or shared (across applications, business units, repositories, etc.) introduces pressure to translate that data in some way to ensure it becomes (or remains) actionable for the recipient (system, team, etc.). These data transformations can introduce errors and delays, which add up as the system’s complexity increases. The pain of this friction is keenly felt in the amount of time it typically takes for transactional data to become actionable insights. Imagine a retailer trying to make as many sales as possible during a heavy holiday season. Things are going well, and the retailer sells most of its local inventory during the course of the day. The data is typically stored in transactional systems and is transferred with some frequency to a centralized warehouse for analysis and reporting.

Figure 1-1. Simplified Business Process: This example product sale flow demonstrates the path data takes across business functions in a modern organization.

Unfortunately, for this retailer, data extraction and loading into its warehouse usually occurs nightly, meaning this valuable insight won’t be available until the next day. Even if the data is available immediately, however, a process or person is typically involved to derive and interpret insights from the raw transactional data. All of these delays mean that the retailer might be unaware until much later in the day that its local storefront needs additional stock, potentially missing out on a large volume of sales. For almost all organizations, the data input at the transactional point of business is raw and constantly changing. Furthermore, in some cases data is manually inputted by a person, which can introduce errors and delays in the process. This means that ingesting, cleaning, analyzing, and presenting the data for insights creates a significant delay in business decisions and action.

Time isn’t the only concern, though. The complexity of the data itself strains organizational expertise. Different areas within the business have different analytics needs. What answers important questions for the sales department will likely not suffice for the shipping and receiving division. Furthermore, the nature of the data (as well as the related insights) varies wildly depending on where in the data path one performs analysis: inventory data, customer data, and production data have entirely different features, formats, and relationships between entries.

As a result, data in the organization will be siloed and require subject matter expertise specific to each area (and the data will often sit in corresponding organizational silos). However, these professionals typically lack the analytical skills and tools needed to derive sophisticated insights from the data. So, they turn to data experts: data engineers, data analysts, and data scientists who conversely lack insight into the unique needs of each line of business. This leaves the organization with a large gap between business expertise and analytics expertise. This “analytics chasm” delays insight generation and wastes analytics expertise on business-as-usual questions.

Technology choices also create difficulties unique to each organization. Recent years have seen a dramatic shift in data management, analytics, and business intelligence technologies and practices with the advent of remote storage and computing. Organizations are rapidly transitioning from locally run servers to cloud-based infrastructure. This means that the data path shown in Figure 1-1 is evolving. New technologies, such as machine learning and artificial intelligence, are being developed to interact with these cloud-based systems. As technology becomes more advanced, industries struggle to employ and retain expertise to keep up with these trends. Additionally, many of these technologies require significant investments of time, money, and labor to implement.

With these concerns in mind, examine how the traditional approach to the data path in Figure 1-1 hinders productivity, delays insights, and prevents the business from competing effectively.

Figure 1-2 illustrates several places within the flow of a product sale that are affected by the inability to directly access the information needed to make intelligent decisions. Each friction point impacts the costs and revenue generated by the company. Together, they make the difference between a company being just functional versus industry leading. How might giving access to analytics help these areas? Let’s dissect this diagram from the perspective of the individuals involved to better understand how these friction points impact the business. At the same time, let’s also look at how access to analytics improves performance.

Figure 1-2. The Data Flow of a Product Sale: Inefficiencies and missed opportunities in data flows become pain points. However, opportunities exist to improve analytics delivery.

The following list takes a deeper look into how the pain points in Figure 1-2 interrupt your business flow and examines how access to data and analytics could lessen that impact:

Suppliers

A great many places in your business rely on data from other areas. This is especially true for those outside your organization on whom you depend. Before a product can be made or a service rendered, your business needs resources. How do you determine which materials you need and how many? You must not only communicate your needs to the supplier but also estimate how other factors could influence those needs. The ability to analyze stock, supply, and demand data enables those responsible for ordering your resources and those providing them to you to monitor and quickly react to changes in the supply chain. This prevents over- and understock issues and may reduce the impact that supply chain problems have on your other business processes.

Production

The people who create or provide your company’s goods and services also need analytics. They must know not only how much demand exists for your product but also how much time, effort, and resources are required to meet that demand. Access to analytics tools and on-demand data would allow them to address business needs more proactively, manage employees more nimbly, and better prevent potential issues from impacting their work. Analytics tools would also deliver insights into how well a production line runs and how it might be improved.

Marketing

This is the leading edge of the customer-facing side of your business. Without data analytics, marketers have few tools at their disposal to understand customer behaviors, determine what customers want across various digital channels, assess which marketing campaigns are effective, and determine how to adapt your product’s pricing, packaging, positioning, and placement to remain competitive in the marketplace. They also need data insights to understand and react to regional preferences and customer demographics. Providing your marketing team with analytics tools and access to data helps ensure that your business appeals to customers, provides the right goods or services, and reaches the necessary markets. The marketing team should have access to behavioral, regional, and demographic data to provide insights back to the company as to what is in demand and how competitive your business is within the marketspace.

Sales

The people who facilitate your customers’ purchasing of your product or service are invaluable. In most organizations, these individuals are given access to transactional systems, allowing them insights into stock numbers, pricing, and local revenue. Is this enough? A sales team with access to analytics could look at buying behaviors, patterns, and shifts in the local store or territory. They could adjust strategies around an anticipated product stock-out or prepare for upcoming demand due to changes in products, services, pricing, or promotions.

Finance

Your finance team requires data to determine how to distribute funds throughout your business and monitor revenue sources. Data insights need to be derived across all areas of the business—from supply chain to employee pay rates and benefits to revenues from sales. These data points coalesce to tell the story of the success or shortcomings of your business. Without analytics tools, financial decisions are typically based on summary values. While this approach meets the basic needs of the business, it does little to anticipate or preempt change. Analytics tools help a finance team go from looking at the now to looking at the patterns leading up to now. The team needs the ability to deeply understand past performance and simulate potential futures to be ready when a financial challenge presents itself.

Executives and shareholders

The leaders of your business need data insights to make decisions on how to move the company forward. Insights delivered to these leaders take time to create, and these delays can have impacts on the decisions made. Moving decisions closer to when events happen is part of the democratization process. It’s a key step to becoming proactive, rather than reactive, with data. Improved analytics capabilities, combined with faster data delivery and cleanup, mean that insights get to leaders sooner.

Customers and prospects

Even your customers and prospects need access to data and analytics. They want to know what items are in stock, how much a given item costs, what delivery or pickup options are available, and alternative options in case their item of interest is out of stock. They want visibility into order status, when it will ship, and, ultimately, when it will arrive at their doorstep or be available for pickup. Today’s customers also expect organizations to reach them on their preferred digital channel of choice with information on upcoming sales and purchasing opportunities. In the experience economy, customers demand that you know what they want, when they want it, and their preferences.

Data also needs to break free from silos within an organization. Cross-functional data access opens the opportunity to generate insights that improve business performance. It’s easy to see how finance units need insights from HR, supply chain, and sales teams, but other areas benefit as well. Sales teams need to know when marketing campaigns influence their business. Marketing teams need to understand the success of these campaigns from the sales teams. Production teams rely heavily on the demand indicated by the sales teams as well as the available resources from the supply chains. Minimizing waste and absorbing demand spikes (as opposed to the more general bottom-line improvement) require your business to leverage multiple (often disparate) data sources to develop holistic insights about critical business processes.

We can dive deeper into how access to data analytics affects your organization by looking at things from an employee’s role. What level of data access does each of your employees need? How knowledgeable are they in the full process of the business, and if they were offered access to analytics, how much skill would they have interpreting the results? Table 1-1 breaks this down for us to examine. It also helps us understand where gaps exist between analytics needs, business knowledge, and analytics tool expertise.

Table 1-1. Employee data needs and skill sets across a typical organization

Employee role	Data and analytics needs	Business knowledge	Analytics tool knowledge
Executive	Aggregate view of the organization; relies on others to generate reports	Top-down view of the organization with an eye to the overall success of the business	Likely is able to view analytics reports, but is probably not familiar with analytics or reporting tools
Middle management	More granular view, usually specific to the area of the business it manages	Strong business knowledge in its respective areas	May have some analytic knowledge but still relies on others to generate needed insights
Business subject matter experts	Need detailed reports to dig into how the business is functioning	Experts in how the business functions	May have some analytics knowledge but still rely on others to generate needed insights
Data subject matter experts	Need full access to data; should have the skill set to analyze organizational data	May have some business knowledge but usually gather requirements for analytics from those closer to the business	Strong analytics skill set, usually with the ability to drill deeper into the data in order to provide insights
Information technology (IT)	Needs the ability to support the gathering, storage, and maintenance of data and analytics tools	Familiar with the business enough to ensure that IT needs are met but probably not integrated in the actual lines of business	Strong technical skills with installing and maintaining analytics tools but may or may not be an expert in how to use them
Lines of business	Data is needed to understand how the business is functioning and make more localized changes	Strong business knowledge, though likely localized to their particular role	Probably little to no knowledge on analytics tool usage and may or may not have access to the data itself
Data scientists	Data is everything they deal with, so they need access and multiple analytics tools	Focus is on data and digging deep into it, but may not be knowledgeable on the nuances of the business	Strong analytics tool expertise, with emphasis on artificial intelligence and machine learning

As we can see from Table 1-1, each role has differing data needs, business knowledge, and analytics-tool skill sets. Those in the business who make decisions and need to derive insights from the organization’s data typically lack the depth of technical skill necessary to generate results using analytics tools. Similarly, those with strong analytical and technical skills typically lack the deep knowledge of how the business runs.

The Issue: Insights Not Available Where Needed

At the root of this complex predicament in which modern organizations find themselves is a simple question: who has access to data, analytics capabilities, and insights? Business leaders are the typical recipients of analytics reports in an organization, but they may not be versed in the skills needed to interpret the results or question the report’s validity. They rely heavily on the strengths of their data teams to collect, analyze, and deliver insights for them to react. From there, the results may trickle down to middle managers but usually never reach the frontline employees. As Figure 1-3 illustrates, there is an “analytics gap” in modern business between those who have the questions and those who have the skills and tools to answer them.

Figure 1-3. The Growing Divide: A chasm exists between those who have analytics expertise and those who have questions.

Additionally, data for many companies is a commodity that is often restricted and protected. Does it all need to be? Security typically assigns permissions based on a mentality of prove that the user needs access rather than prove there is a reason to restrict access to this data. As a result, many functional areas within the company that could benefit from analyzing data are restricted from accessing data in the first place.

What does this restriction of data access mean for your company? According to Dresner Advisory Services, 50% of all businesses have some sort of lag in their data path, and at the same time, 50% claim they make data-driven decisions.¹ There is a correlation between these two facts. The faster the data path, the more likely a company is to make data-driven decisions. Dresner also indicates that 77% of companies expect their data frequency requirements to increase in the future. So, if your company wants to be a leader in your market, you want to get ahead of this need. If you want to make real-time decisions based on data, then you need the capability to analyze this data as close to real time as possible.

The typical manual processes used by most organizations make this scenario appear inevitable. Is there any way to enable all layers within an organization to utilize data effectively and safely? This is the vision of analytics democratization.

A New Approach: Democratizing Analytics

Democratizing analytics means making both data and analytics capabilities available at all levels of the organization. To achieve this, organizations will need to look for intuitive, easy-to-use analytics platforms that are accessible to any skill set—from novices to experts. In short, it’s data, and the ability to draw insights from it, for anyone at any level of the organization—from leadership to marketing to sales. This removal of obstacles preventing the delivery of insights from data makes the company more nimble and faster to react to growing change and increasing competition in the marketplace.

Analytics democratization is made possible through a combination of process and technology innovations. On the process front, democratizing analytics requires implementing a standardized, governed approach to sharing data and analytics capabilities with those who could benefit from it. This doesn’t mean that data just goes to everyone haphazardly. But it does mean that we need to shift our mindset and approach from a data-hoarding to a data-sharing mentality. Regarding technology, democratizing analytics requires user-friendly, flexible, analytics tools with reusable and repeatable workflows to enable the new data beneficiaries.

The ability to analyze data at any level of the organization means that employees at the middle management and business operations levels are able to draw and interpret insights from data. Being closer to the business operations means that these professionals can combine data-driven insights with functional knowledge of the business. Combining these two things generates opportunities for continuous strategic improvements at small and moderate scale. It also allows those employees closest to the customer to improve the customer experience with data-driven insights.

Broader analytics access means faster insights and more innovation.

When companies provide analytics capabilities and data access at all layers of business, they open the doors to innovation, faster insights, and freedom to develop advanced analytics to make more intelligent business decisions and be better prepared for the future. These organizations that democratize analytics position themselves to become leaders within their industry.

¹ Dresner Advisory Services, “Data Latency Remains a Challenge as Data-Frequency Requirements Increase”, December 14, 2022.