Semantic Modeling for Data

Chapter 4. Semantic Model Quality

Come, give us a taste of your quality.

William Shakespeare, Hamlet

The whole goal of this book is to help you build and use high-quality semantic models, so a question that naturally arises is how you can measure this quality. For that, in this chapter, I describe the main quality dimensions that you should consider when evaluating a semantic data model, along with basic metrics and measurement methods for each dimension.

Before we dive into the concrete dimensions and metrics, it’s important to understand that there are two different approaches of measuring the quality of a semantic model. The first approach is called application-centered and measures the improvement (if any) that the usage of a semantic model brings into a particular application, such as a semantic search engine [62] or a question-answering system [63]. In doing that, it typically compares the application’s effectiveness before and after the incorporation of the semantic model.

The advantage of this approach is that we can immediately see if the usage of the model has a visible benefit to the application, and thus assess directly its fitness for use. There are some drawbacks, though. First, the observed absence of a such a benefit does not necessarily that the semantic model is low quality; the problem can be with the way the application uses the model as well. Second, even if the problem lies in the model, the end-to-end quality score does not really tell us what is wrong ...