Java: Data Science Made Easy

Text features

In the previous chapter, we learned a lot of things that could be applied to text data and used some of the ideas when building the search engine. Let's take these features, include them into our model, and see how our AUC changes. 

Recall that we previously created these features:

• Cosine similarity in the TF-IDF space between the query and the text fields of the documents, such as the title, the body content, and the h1, h2, and h3 headers
• LSA similarity between the query and all other text fields

We also used GloVe features in the previous chapter, but we will skip them here.In addition we won't include the implementation of the previous features in this chapter. For information on how to do it, refer to Chapter 6, Working ...