Hands-On Software Engineering with Golang

Generating convincing summaries for search results

Generating a descriptive short summary that conveys enough information to the user about the contents of a web page that matched their query is quite a hard problem to solve. As a matter of fact, automatic summarization is an active research field for natural language processing and machine learning.

Arguably, building such a system is outside the scope of this book. Instead, we will be implementing a much simpler algorithm that yields plausible summaries that should be good enough for our particular use case. Here is an outline of the algorithm's steps:

1. Split the matched page's content into sentences.
2. For each sentence, calculate the ratio of matched keywords to the total number of words. ...