Changing Realities for Business

Chapter 4. Building Out the Augmented Reality and Internet of Things Layer

Although Chapter 3 and this chapter both have case studies, Chapter 3 shows how companies are currently using augmented reality (AR), whereas this chapter looks at how to integrate new tech with old machines and ways of working. It’s a topic that’s important for a book that speaks to the business of AR; if you’re working inside an older or a more traditional business (or selling to one), half of your job is building the technology—the other half is explaining it and having it accepted.

This advice and insight can help.

Using AR to Help Visualize Sensor Data

Not only is the history of AR-enabling devices like tablets and wearables systems important to discuss—there’s an evolution of sensors that is relevant, too.

The kinds of sensors we have in modern consumer devices were once so expensive that only NASA could afford to invent (as well as build) them. If you look around the world today, many things are sensor-enabled or controlled—consider that every automobile is filled with hundreds of sensors, for example, one for the crankshaft, one to measure fuel vapor pressure, one for vehicle speed, one for camshaft angle, door ajar, seat pressure, seatbelt, spark knock, and on and on. It’s actually really interesting the number and variety of sensors that exist.

Here’s a primer for those who are new to how all this works:

Walk onto an ordinary factory floor and you’ll see dozens of sensors on any given machine. Some sensors are tracking things like vibration of multiple axes. Other sensors are monitoring pressure, temperatures, or mass flow. There are variable frequency drives controlling motors. And all of these sensors are less expensive and smaller than ever. This has led to an explosion of new sensors in the marketplace, and more of them are coming and being added all the time. Why is that relevant?

Sensors on industrialized, instrumented machines are collecting raw data. And that data, when used properly, is analyzed and then given back out to the worker in various ways—most usefully, in various forms of visualized data to help those workers do their jobs better.

That is what augmented reality does right now.

IoT Can Support AR and Vice Versa

Augmented reality and the Internet of Things (IoT) are complementary tech spaces that will be increasingly intertwined as both develop.

Here’s how the two fields link together: AR gives you the ability to visualize, share, and contextualize data coming from sensors on machines, automobiles, refrigerators, wearable devices, environmental tracking devices, and electronics, or anywhere else sensors are found.

Augmented reality, at its best, is fed by the systems that collect sensor data—the basic idea behind the IoT. This allows real-time information about things like the maintenance of machines or the shelf-stock in stores to be fed back to the people responsible for caring for them.

This is the future of how these two spaces work together.

The rub: because IoT systems are still being implemented and refined, not every company that has access to this data knows how to access or use it.

Still, it is worth mentioning here. And here’s why:

Experts who have already made inroads selling IoT products are essentially advance scouts for the AR industry. They are already on the leading edge of selling in new technology to traditional industries. They are ushering in the step that logically precedes AR adoption. And as such, they have valuable advice about how to make the case for implementing AR.

That’s exactly why I sought out an IoT expert for his perspective on how to sell AR into traditional industries, who to speak with, and when to approach.

Zach Supalla is the CEO and founder of Particle.io. His company is an IoT platform that offers companies a turnkey system to make their business IoT-enabled. What does this look like? Particle.io provides the sensors and hardware that add connectivity, the software that connects to and controls the equipment, the cells and SIM cards that bring devices online, and a way of reading and using the data that comes back. His customers are all across the globe, of every size, and span across all industries. They include OEMs, startups, people making industrial equipment, consumer appliance companies, hot tub manufacturers—a wide range of industries. He has more than 100 developers on his team and ships to 175 countries, which is to say that he has plenty of experience and great insight.

Supalla’s basic mission is to give companies that make products, but have limited engineering researchers on staff, the ability to get things online and collect data. And even though his platform doesn’t take this to the next step—using AR to visualize the data that comes from the machines—he does have experience that is highly relevant here. Supalla is an expert at talking with companies about the benefits of new technology, some of whom might have little understanding of it at the outset.

As he says, the most common profile of his customers is “people who have been making stuff for 50 to 100 years. They [are companies who] outsourced a lot of their electronics design to Taiwan in the 90s. They might have a $100 million product a year, but they only have two engineers on staff. And there are a lot of problems; how do you connect legacy systems to IoT? There are a lot of very expensive machines it doesn’t make sense to replace; you just want to add a little connectivity.”

According to Supalla, the argument is the same whether you’re selling in IoT, machine learning, or any other new tech within an older industry. And the way you make that sell—internally, as a CTO or externally, as a vendor—is the same, whether you’re working with AR, IoT, or any other software. It’s the goal that drives willingness to adopt and the way you frame your argument, not the technology itself.

Here’s what Supalla had to say about when and how to sell in emerging tech:

The way you make your argument and where you sell it in within the company, essentially, comes down to what you need to do. Are you trying to convince your company to go through what amounts as a major business transformation? Or are you just trying to get buy-in on a piece of software that’s going to make people’s lives marginally better in some narrow but in a specific way? [Are you] trying to sell somebody a piece of software that they’re going to pay 50 bucks a month for, that they install in 10 seconds and just has some minor impact? Or [are you] selling somebody a multimillion-dollar transformative new business model?

You could be doing that with AR or machine learning or any sort of major technological transformation that applies to these older industries.

That’s what makes it different, not the technology itself.

It’s a big deal in terms of learning curve and investment in order to take on any new technology. So who are the companies that are willing to undergo the expense? Supalla’s take is this:

What I see are a few patterns of the kinds of companies that are willing to kind of go big with technology in spaces that traditionally haven’t. One pattern is a fast-follower pattern, which is that one of your competitors already did this, and now you feel like you’re chasing. No one likes to be in that position, right? You don’t want to be a second mover. But if you are the second mover, you also have a tendency to try and leapfrog the person who is out in front. You don’t want to just be always one step behind them. You want to say, “OK, well, they did this thing, and now I want to do it two times better, so that I can get out front and be the market leader.” So that’s common.

Another big pattern [is] with family-owned businesses that are passed down to a younger generation, where you have a business passed from father to son and the son wants to go bigger than dad or put his stamp on it—and also is potentially more likely to be technology-minded in terms of his approach. That’s a pretty common pattern.

Similarly, when new leadership is brought in, it’s sort of a variation on a theme. If a company is bought by a private equity company, and the executive team is booted and replaced with a new one, or just, you know, the CEO retires and the lieutenant is promoted to be the new executive. Any one of those is a good reason to believe that company is about to go through a major transformation.

With that in mind, my big question for Supalla was: is it even possible to sell in a big new idea or technology unless you’re speaking with the C-suite or CEO? Can you sell in these ideas if you are someone who is within a company at a lower level? Or if you are selling to someone within a company at any level other than CTO? Supalla says yes. But if you do, there are a few guidelines to choosing the right person within the company:

You end up needing to either pick somebody who is very influential—and that requires digging in and understanding internal politics within a company, which is often necessary for these kinds of sales—or scoop somebody’s existing budget.

With IT transformations, there’s often the torch-bearer of the project who is an up-and-comer within the industry. Maybe someone who is [newly] hired, maybe a frontline or a junior manager who has really impressed everybody, who is rising through the ranks. Someone that has been flagged as high potential for a future executive position. That person then carries a lot of weight and is able to be even potentially more influential than their job title would suggest.

If that person says, “I have a big project, and I have something big and important,” that can also be a opportunity for them to shine. They see [implementing an AR/emerging tech project] as a way for them to shine. Or their manager or the leadership team says, “This is a great way for them to really prove their worth.” That’s one common pattern we see.

The other way to implement a new tech project like AR (or sell it in to a company) is working it into an existing budget. If you’re working with someone who has hundreds of thousands of dollars to spend on software in any given year, you can find ways to fit your solution or technology into that budget. That way, it doesn’t need to be considered a business transformation or require C-level sign-off. Supalla calls this a “land and expand” approach:

We’re just going to do this pilot or proof of concept, but we’re going to use it as a way to get in, to get visibility to the C-level executives, where the second phase of this is going to be the big transformation.

Finally, I asked Supalla about augmented reality and what kind of transformation one of the big conservative companies he works with might expect from the technology. Because although he plays all aspects of the IoT stack, right now in manufacturing and industry, AR is not that different. Supalla explains:

I can’t give specific examples because any example I would give would be a little too obvious to figure out who they are, and most of our customers are pre-production, but I can give generalisms…One of the examples I see [that’s] really relevant with ARs is factory floor, and the end state that I think is really compelling to a lot of shop floor managers is the idea that you are going to be able to put on a headset and you’re going to be able to walk around your shop floor and you’re going to give this kind of heads-up display, that shows this information coming from all your machines, that shows their yield, it shows the errors coming out of the machine, etc.

This is, of course, [something] I think about a lot, because it’s very much an overlap between IoT and AR. It’s a case where the information that you need to pull from these machines is the IoT side, and then the way that you present it is the AR side.

The technology to bring machines online and present information from them exists. The technology to take that information and put it in [the] form of a heads-up display exists. Things like Microsoft HoloLens and the other variations of AR display systems exist, not to mention the fact that you can just do it from a lot of smartphones. All the pieces are there, but each individual thing hasn’t been instrumented yet in a way that allows for that full experience to exist.

No one is doing for AR yet what Supalla and Particle.io are doing for IoT. But one team has developed turnkey AR for integration into the technology stack. This is a huge key to moving the technology forward. It’s a big deal, and no one has reported on it yet. The next section explains why.

AR Tools Are Becoming Embedded in the Technology Stack

While writing this book, I kept looking for a major advance in AR and the key to more mass adoption. Although Apple’s work helped to spark that in the consumer world, in the industrial world, there needed to be something else to make more companies want to adopt the technology and make it easier for them to do so. It was a missing link I’d been looking for.

PTC’s Thingworx is the leading industrial IoT platform, used for smart connected operations. It allows you to integrate and structure data from all different sources, including sensors, machines, CAD files, and business systems. It can also easily turn all that data into AR tools. It is a comprehensive and complex system—of which AR is only a small part of the story. It is the ending point of a massive IoT data funnel.

DAQRI, being focused specifically and only on AR, takes a far more direct route—both in its creation, and in making it part of business systems—something I discovered only when interviewing company co-founder Gaia Dempsey.

Here’s what she had to say—and why this is such a big deal.

Case Study: DAQRI

As a company, DAQRI is best known for making a $17,000 head-worn AR device used by some of the biggest manufacturers around the world. When Dempsey and I spoke, we talked about DAQRI’s history in AR and how it’s grown to a staff of 300. We talked about the manufacturing process as a whole (information that is very interesting if you don’t have a working knowledge of how products are made). And then Dempsey moved into the conversation that felt like a massive “Aha!” moment.

The reason: Dempsey and DAQRI created the technology as well as the business parterships to integrate AR tools and capabilities into some of the biggest product lifecycle management (PLM) systems in the world—the plumbing behind how most products are made in industries like automotive and aerospace. These systems automatically connect to the CAD renderings you need to build manufacturing AR visuals in the first place.

It’s the answer to the question: how do you make this turnkey for companies to adopt? And it is one key to moving AR adoption forward.

And even though no one is writing about it (or seems to have noticed), it is one of the most important advancements in the AR world.

It’s not a technology at all; it’s automatically embedding AR in the technology stack.

For AR to become an ordinary part of how we conduct business, it must be built into workflows and production processes in ways that become second nature. What DAQRI built bakes AR into the plumbing.

The interview that follows is how Gaia Dempsey explains it. Our conversation began with an overview of the manufacturing. Although some of this information might not be new to those in manufacturing, it will be new to everyone else. And for manufacturers, it gives new insight into how AR interweaves with the entire system—in what I am seeing as a revolutionary way. This is one piece that has been missing thus far in spreading AR far and wide.

Dempsey explains more, beginning with a necessary primer on how manufacturing systems are organized (you’ll see why it matters shortly):

What we’re doing, with Siemens is tool chain integration…[for] a PLM system.

Anybody who works in manufacturing [will] have a PLM system; it works in conjunction with an environment resource planning system. Basically, these are the core business systems that organize [and] manage all of the assets and resources of the business that is actually going to produce something.

The data that gets stored in a PLM is all of the CAD models. It is where you have your bill of materials—you keep the record of what you’ve actually built. It’s where you keep all of your electronic work instructions, if you use electronic work instructions.

For example, you can imagine the complexity of the PLM system within BMW. Cars are in production for five to seven years typically. There’s dozens of different models. Even when a particular model is no longer in production, that information doesn’t go away because you still have all of the information to keep a record of—showing what went into that car and how it was put together. That becomes the single source of truth for your entire engineering and production processes. After the car isn’t in production anymore, those data points are still important for maintenance because the life of the car could be 20 years after that. It’s a deep rabbit hole to go down. These enterprise systems [are] the plumbing [that maintains those records].

The reason I talk about this is we’ve integrated the DAQRI platform with a PLM, which is called Teamcenter. It is one of the most widely used PLM tools in the world. BMW uses the Teamcenter PLM in their build process for…all of their cars. SpaceX uses the same thing, the Teamcenter PLM, for building rockets. Northrop Grumman [the aerospace and security company] uses it to build all of their projects. Siemans [uses it too].

A good, flexible PLM will enable you to do large-scale production and, to some extent (and not all PLMs are this flexible), you’ll also be able to do small-scale prototyping. For example, with Northrop or a SpaceX you’re actually not building 100,000,000 of every unit. You might be building five.

It is transformational for these companies and for our customers to suddenly have a pathway to rapidly adopting augmented reality in their day-to-day operations. All of their existing plans of record…all of their existing CAD modeling and electronic work instructions…all of their maintenance records…all of these other things are already in their PLM. They might have literally years or decades of data that they have been evolving and working on and contributing to and refining in that system. And so [it makes it easy] for a large entrenched business to say, “Oh yeah, we understand that augmented reality can transform our business model. We understand it can make us so much more efficient.”

In addition to the Teamcenter integration, Dempsey says the company is also integrating with Autodesk BIM 360 for the construction industry, IBM’s Watson (a key artificial intelligence platform), and Siemens XHQ, which is widely used in the oil and gas industry. It is a key development in the history of AR not only because it makes building and implementing AR systems much easier, but also because it opens a new path for manufacturers to streamline their businesses. The reason: manufacturers that have been using Lean business practices have gone as “lean” as they can. In some industries, the entire manufacturing process is flexible, changeable, and entirely streamlined. What this means is that they need some new ways to improve productivity and decrease cost. Dempsey puts it this way:

In the 1950s and 60s, everybody thought the factory of the future was one that was 100% automated and had no people in it. What we realized is that the mass market cannot support just billions of widgets that are all the same…and that’s what an entirely automated factory would do. It hardens, it doesn’t have flexibility, that’s just not what our economy supports. In fact, we have the opposite trend—more and more customization, more and more personalization.

The reason lean manufacturing was different is that you had more of a partnership between people and machines. Yes, you still have machines in the process but you also have the intelligence and the flexibility of people. For instance, one single robotic arm can be designed such that it had a huge amount of flexibility in what it could do and the person would program it to do different things.

With greater need than ever before to compete on price, customization, and efficiency, you can use things like IoT and AR to give you company a boost. When those two technologies combine, they become even more powerful.

Imagine that you have a young factory worker sent to repair a jet engine made in 1982, when she was born. By using AR—that pulls in CAD illustrations from a PLM system (that keeps historical records)—she can know exactly how the that model of engine was built, though she’s never seen one before. And she can have precise instructions that allow her to complete the repair quickly and perfectly.

Because of the unique characteristics of AR, we know that it accelerates learning. We know that it helps you to retain information longer. We know that it improves performance, and we know that it also levels the playing fields. Dempsey recommended a research paper from 2015 called “Using Augmented Reality to Cognitively Guide Assembly” by Lei Hou, Xiangyu Wang, Leonhard E. Bernold, and Peter E.D. Love that speaks to some of this. It actually can minimize or even eliminate individual differences in cognition and spatial reasoning. Things like giving people to ability to do complex tasks equally well, no matter your level of experience or your level of natural abilities for different types of cognition and reasoning.

As Dempsey and stories from the Index AR and Scope AR case studies show, we also now know how to communicate information in AR so brains perfectly understand it and there is no data loss; there is no loss in translation between either spatial orientation placement or scale. You can essentially create a process by which the person makes zero errors through augmented reality. They’re as accurate as a robot would be because you’re presenting the information to the brain in such a way that we can execute against it perfectly. Dempsey expounds:

The biggest part of our engagement with our customers is all around these issues. We understand deeply, “Here are the different trends we know your consultants are telling you to care about but you don’t have a strategy for it yet.” [There’s a great] research report that The Economist did a few years ago where they surveyed all of the top industrial businesses in the world and some huge percent, like over half of them, were collecting data but didn’t have a data strategy. Or didn’t have an IoT strategy. [Note: The Economist has published an updated 2017 version of this report here.] So, it is definitely an emerging area. I think there are players that are getting smarter and that are really leveraging those capabilities.

I would point to GE and the Predix Platform as a really great exemplar of how a traditionally industrial manufacturing business is now becoming positively transformed by Silicon Valley trends. They now have 400 software engineers that are just writing code, essentially to take the massive amounts of data that they collect and generate insights from that data. Then [they] actually create a platform on that and provide that as a service to their customers. That’s another really big trend: manufacturing companies are no longer just manufacturing companies. They’re all services companies, as well, in many ways.

The roots of this [service] trend actually started a long time ago. In the 1980s, Rolls Royce started offering power by the hour. Instead of selling you a jet engine, you pay for flight time by the hour. And that, of course, had higher margins but it also transfers the responsibility for the maintenance and care of that engine to the original manufacturer, which changes, in some ways, the liability and changes the responsibility and the business model. So then Rolls Royce cares more about the longevity of that product and has developed huge services segment of their business to do preventative maintenance.

As Dempsey and I discussed, there is also another big trend around essentially, predicting failures before they happen, and IOT is one of the driving forces behind that, as well. The question is this: can AR play in all of these trends in different ways at different levels? Or, is AR as a tool just one discrete, single-use piece in the long continuum and evolution of manufacturing? I know AR can become the user interface for the industrial internet space. Because of the way it can visualize information, it has the potential to play a huge role in IoT. What else can it do, though? Here’s what Dempsey had to say about that:

AR also plays a huge role in the trend of manufacturing to help service providers, because it meets the need to train field service workers—one of the big challenges. When you go from a manufacturing company to a field services company, you’re providing services and maintenance. The same thing is happening in construction—now you’re not just an architecture firm or a construction firm, you also do facilities management and you care for the life of the building over 30, 40, 50 years.

What does that mean? It means that your inventory of things that an employee has to reasonably know has just exponentially increased. They might be sent out to do a repair on something that’s 20 years old and that they’ve never worked on it before in their life.

What AR gives you is this connecting thread that allows you to pull down data when and where it’s relevant and relay it in such a way that the job is done perfectly by someone, without having to have had 20 years of experience. It pushes and democratizes decision making to the lowest echelons, which is also the trend that everyone is going after. It is empowering decision making at the edges of the network both in terms of the way that IOT structures are created from a technical perspective and pushing computing and processing to the edges of the network. We are moving away from centralized command and control. The old model doesn’t have the ability to operate in our current environment anymore.

The same thing [is] going on in the military. We’re now doing several military projects [where] this is exactly what they’re working on. They’re trying to figure out how do we actually empower people to make decisions and to give them the information they need in real time to make good decisions? Sending out an email doesn’t do that. We’re looking for new tools and AR is going to play a huge role in that.

And that goes across the factory floor, it goes across drivers driving, it goes across medical allocations in surgery or ER, emergency medicine, first responders.

This is the power that AR has.

The Business Justification for AR (aka Show Me the Numbers)

Both Supalla’s work and Dempsey’s point to the fact that AR technology can be useful, adopted, and sold in to even the most traditional companies, or the largest. However, there is a question to address, especially if you in a big company or a position in which you must make a case to others. How much return could there be for you by going into AR? And its adjunct: what financial numbers exist to show that this can be truly valuable?

Matt Sheridan from PTC has an answer on how to figure that out. His is an interesting approach to answering. And because he has literally written the book on giving effective sales engineering presentations and demonstrations, entitled A+ Demonstrations: Excellence in Sales Engineering (Boston Writers Publishing). He is the perfect person to speak to the topic.

His advice: because AR is still relatively new in large-scale rollout and the companies that have done that aren’t sharing numbers publically, you essentially back into the numbers. Here’s is the example Sheridan gives and how he demonstrates AR’s value:

There’s a study that was by a partner of ours called ServiceMax. The study is called “A Diamond In the Rough: Unleashing the Power of Field Service Transformation.” The study talks about two aspects of service. First, that service is continuing to grow—in the sense that there’s more service taking place—and various companies are looking at service as a revenue stream. Before, service was simply the add-on for selling a product. Now, companies are actually selling the service itself as part of their offering. Therefore, revenue is increasing.

If you look at your service department, most good service organizations have some metric that they’re tracking. First-time fix, mean time to repair, or mean time to install. If you were going to have a conversation about how AR might improve a particular area and you wanted to tie that to something, let’s take a look at first-time fix, which is about whether an issue can be resolved on the first try. [Note: first time fix is also a metric often used in IT.]

Referring back to that ServiceMax whitepaper, it talks specifically about service of two companies. Company A has a first-time fix rate of 88%, considered at the high end of the first-time fix rate range. Company B was at 10%, at the low end of the first-time fix rate. If both companies are doing 400 service jobs a day, the first-time fix rate differentiation means Company A is able to fit in 100 more jobs a day. Multiply that by 250 days a year, that’s 25,000 extra jobs. So for Company B to compete with Company A, they need roughly 25 more people to complete all those jobs. I then looked at Payscale.com and looked up the average salary of a service technician in North America and found it to be $55,000 a year. Simple math—25 times $55,000 a year gives you $1.3 million investment for Company B to catch up with Company A. That gives you a ballpark number.

What that means: if you find what’s causing the first-time fix rate problems, and what’s causing increased cost, then you can tie that to how AR can help solve the problem.

It’s a little bit of A equals B, B equals C, therefore A equals C. We don’t have—at the moment—the exact company that says “I saved $1.3 million with AR.” But you can see where the conversations should be happening. And people like real numbers, so they’ll have those conversations.