Data Makes the World Go 'Round

Chapter 6Metadata: What It Is and How It Is Evolving

In this chapter, we will examine the following:

• Metadata, metadata, metadata
• Tools and practices for metadata management
• Metadata evolves
• The metadata platform

I’ve talked about new data types and some new data architecture patterns. I’ve also introduced how AI is being infused into data and analytics software products to make them easier to use. Throughout this I’ve mentioned and defined metadata and the importance of it. In this brief chapter, we’re going to look at metadata, what it is, how it has evolved, and how it continues to evolve.

Metadata, Metadata, Metadata

As I’ve discussed, metadata is data about data. When data professionals think about metadata, they often think about three kinds: technical, business, and operational metadata.

Technical metadata describes the structure and the format of the data that is stored in systems such as a database or data warehouse. This includes information about the kind of data it is (integer, string), table and column names, schema definitions, file size and format, and so on. Historically, technical metadata has been used by data engineers, architects, and IT professionals who are responsible for building, managing, and maintaining the organization’s data infrastructure. They rely on it to understand how data is structured, how it flows between systems, and how to ensure it is processed efficiently.

Business metadata, on the other hand, is what it sounds like. Business metadata ...