How intelligent data platforms are powering smart cities

Smart cities and smart nations run on data.

By Ben Lorica
October 14, 2015
Marina Bay and the skyline of the Central Business District of Singapore at dusk, by William Cho. Marina Bay and the skyline of the Central Business District of Singapore at dusk, by William Cho. (source: Wikimedia Commons)

According to a 2014 U.N. report, 54% of the world’s population resides in urban areas, with further urbanization projected to push that share up to 66% by the year 2050. This projected surge in population has encouraged local and national leaders throughout the world to rally around “smart cities” — a collection of digital and information technology initiatives designed to make urban areas more livable, agile, and sustainable.

Smart cities depend on a collection of enabling technologies that we’ve been highlighting at Strata + Hadoop World and in our publications: sensors, mobile computing, social media, high-speed communication networks, and intelligent data platforms. Early applications of smart city technologies are seen in transportation and logistics, local government services, utilities, health care, and education. Previous Strata + Hadoop World sessions have outlined the use of machine learning and big data technologies to understand and predict vehicular traffic and congestion patterns, as well the use of wearables in large-scale health care data platforms.

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more

As we put together the program for the upcoming Strata + Hadoop World in Singapore, we have been cognizant of the growing interest in our host country’s Smart Nation program. And more generally, we are mindful that large infrastructure investments throughout the Asia-Pacific region have engaged local leaders in smart city initiatives. For readers comfortable with large-scale streaming platforms, many of the key technologies for enabling smart cities will already be familiar:

Data collection and transport

In smart cities, Internet of Things and industrial Internet applications, proper instrumentation, and data collection depend on sensors, mobile devices, and high-speed communication networks. Much of the private infrastructure belongs to and is operated by large telecommunication companies, and many of the interesting early applications and platforms are originating from telcos and network equipment providers.

Data processing, storage, and real-time reports

As I noted in an earlier article, recent advances in distributed computing and hardware have produced high-throughput engines capable of handling bounded and unbounded data processing workloads. Examples of this include cloud computing platforms (e.g., AWS, Google, Microsoft) and homegrown data platforms comprised of popular open source components. At the most basic level, these data platforms provide near real-time reports (business intelligence) on massive data streams:

Real-time Data Fusion.
Figure by Ben Lorica.

Intelligent data applications

Beyond simple counts and anomaly detection, the use of advanced techniques in machine learning and statistics opens up novel real-time applications (machine-to-machine) with no humans in the loop. Popular examples of such applications include systems that power environments like data centers, buildings and public spaces, and manufacturing (industrial Internet). Recognizing that future smart city applications will rely on disparate data — including event data (metrics from logs and time-series), unstructured data (images, audio, text), and geospatial data sources — we have planned sessions at Strata + Hadoop World Singapore that will cover advanced analytic techniques targeting these data types.

Smart city platforms represent some of the more exciting and impactful applications of real-time, intelligent big data systems. These platforms will confront many of the same challenges faced by applications in the commercial sector, including security, ethics, and governance. At Strata + Hadoop World Singapore, we’re looking forward to highlighting the intersection of communities and technologies that power our future cities.

Related content

Post topics: Data

Get the O’Reilly Radar Trends to Watch newsletter