How the Internet of Things impacts big data strategies

Use cases from Harley Davidson, Google, and Barcelona.

By Jim Scott
November 17, 2015
Arching fountain of a Pahoehoe. Arching fountain of a Pahoehoe. (source: By J.D. Griggs on Wikimedia Commons)

What exactly is the Internet of Things? Put simply, the Internet of Things (IoT) connects devices such as everyday consumer objects and industrial equipment to a network. Software is used to manage and gather information about these devices in order to increase efficiency, enable new services, and achieve other kinds of health, safety, or environmental benefits.

Four main categories

When we think about IoT in terms of the user, we can group it into four main categories: personal, group, community, and industrial.

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more
Image courtesy of MapR.

Personal IoT sets the focus on a single person. For example, users might be wearing a smartwatch (such as Android Wear or Apple Watch) that measures heart rate, which becomes a piece of data that can be shared with the user’s doctor.

Group IoT extends its scope to a fairly small group of people, such as a family. For example, in the context of a smart home, sensors capture temperature and lighting conditions for optimal comfort. In a car, group IoT is made up of the people inside it, and may include the same types of sensors found in a smart home.

Community IoT takes into account a larger group of people (thousands or more). For example, public infrastructure, such as smart cities may contain sensors that track cars and road conditions to help maintain a smooth and efficient traffic flow. Another example is found in health care. A patient’s heart rate, body temperature, and blood pressure may be monitored 24/7 and their health status made available to the medical staff.

Industrial IoT is the most established branch of the Internet of Things. It features examples ranging from individual companies operating a smart factory to various organizations working together, as in the case of a retailer supply chain.

The third wave

The Internet of Things is emerging as the third wave in the development of the Internet. As a point of reference, the first wave was actually getting people connected to the Internet; the second wave was people “going mobile.” Cisco predicts that connectivity costs (for businesses and consumers) will decline by 25% annually between 2013 and 2020, while the annual growth in the number of connected objects will grow by 25% during the same time period. Based on those assumptions, we can expect 50 billion connected things by 2020, and the digital infrastructure is growing rapidly — actually 5x faster than electric and telecommunication industries.

This rapid expansion of smart objects will impact our personal lives, workplace productivity, and consumption. In addition, there will be a string of new related businesses that will expand the Internet “pipes,” those that will analyze streams of data, and others that will make new things we haven’t even thought of yet.

IoT use cases in deployment

Harley-Davidson Inc. has recently launched a new manufacturing facility in which every piece of equipment is a connected device, and every variable is constantly measured and analyzed. Software at their plant in York, PA, keeps track of how different pieces of equipment are performing — measuring the temperatures and speeds of fans in a painting booth, for example. If the software detects that a measurement has deviated from the acceptable range, it will automatically adjust the machinery.

Image courtesy of MapR.

All of these data points come together in SAP Connected Manufacturing, which integrates the manufacturing results with real-time feeds into the SAP HANA platform. This process makes it possible to perform predictive analytics, which drives cost efficiencies and achieves higher asset utilization. The result? Harley-Davidson can now build 1,700 motorcycle variations on one production line, and ship a customized motorcycle every 90 seconds. In addition, the company can now make deliveries within just six hours instead of 21 days.

Self-driving cars use many sources of data, from lasers to stereo cameras to windshield wiper rain detectors, to figure out what’s happening around them. Google’s self-driving car, for example, gathers 750 megabytes of sensor data per second. It captures everything it sees that is moving, from trucks to pedestrians to rolling balls. The car uses personal computers that process 1.3 million laser measurements and make 20 driving decisions every second. Google doesn’t have the only set of wheels on the IoT highway — six automakers recently announced plans at the Consumer Electronics Show (CES) 2015 for autonomous driving vehicles that will be on the road between 2017 and 2020.

Self-driving car
Image courtesy of MapR.

Barcelona has embraced the Internet of Things, creating a smart city to stimulate the local economy, improve communications among city departments, and minimize government costs. One citywide improvement relates to traffic and parking — in the past, 40% of city traffic was caused by drivers looking for parking. Now, in-ground parking sensors communicate with in-car devices to quickly direct cars to open parking.

Barcelona has also transformed the experience of waiting for a bus; now, riders use “smart” bus stops and buses to find route information, as well as information about the area, via smart panels located on buses and in bus shelters. The city has also installed highly efficient street lights, which are dynamically managed in order to save energy and optimize maintenance — and the street lights brighten whenever they sense pedestrians.

Image courtesy of MapR.

The Australian government funded Smart Grid, Smart City, a $100 million project called Ausgrid, which tested a range of smart grid technologies — gathering information about the benefits and costs of implementing these technologies in Australia. The project was used to provide near real-time information about power use to help customers control costs. In addition, detailed real-time data about the city’s power supply meant that potential equipment faults could be proactively identified and repaired to avoid power interruptions. Up to 30,000 households participated in the project (which ran between 2010 and 2014). 

Smart Grid, Smart City
Image courtesy of MapR.

The Ausgrid project, overall, provided a safe and reliable electricity supply to more than 1.6 million homes and businesses across the Sydney area, and led a consortium of partners working together on this trial. EnergyAustralia, the Smart Grid, Smart City retailer partner, tested innovative technology and pricing offers. These products also made the most of new smart meters and were designed to give homes greater choice and control over their bills.

Data that falls into the IoT category tends to be time-series based data. That is to say, the data coming from sensors is specific to a sensor at a point in time. Storing the data for long-term usage and analytical purposes can be rather cumbersome. Fortunately, there are databases that are dedicated to storing time series data — OpenTSDB is one.

When running a database like this, the first concern is usually how fast it can ingest data. Since this type of database scales linearly, it is possible to ingest data at a rate upwards of 25 million data points per second per server in a cluster. With 10 servers in a cluster, it can handle 250 million data points per second. OpenTSDB is a good way to get started with time series data, and it also comes out of the box with the ability to downsample and perform different types of aggregations (avg, sum, min, max).

There’s no doubt about it — the Internet of Things has moved into the mainstream, and there’s great potential as companies store, process, and analyze IoT data and turn that data into relevant business insights.

Post topics: Data