Natural Language Processing with Spark NLP

Chapter 13. Building Knowledge Bases

This application is about organizing information and making it easy to access by humans and computers alike. This is known as a knowledge base. The popularity of knowledge bases in the field of NLP has waned in recent decades as the focus has moved away from “expert systems” to statistical machine learning approaches.

An expert system is a system that attempts to use knowledge to make decisions. This knowledge is about entities, relationships between entities, and rules. Generally, expert systems had inference engines that allowed the software to utilize the knowledge base to make a decision. These are sometimes described as collections of if-then rules. However, these systems were much more complicated than this. The knowledge bases and rule sets could be quite large for the technology of the time, so the inference engines needed to be able to efficiently evaluate many logical statements.

Generally, an expert system has a number of actions it can take. There are rules for which action it should take. When the time to take an action comes, the system has a collection of statements and must use these to identify the best action. For example, let’s say we have an expert system for controlling the temperature in a house. We need to be able to make decisions based on temperature and time. Whenever the system makes a decision to toggle the heater, or air conditioner, or to do nothing it must take the current temperature (or perhaps a collection ...