O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Data

In order to build our search and retrieval system, we will use the same data for training and testing. We will use the Quora duplicate questions data for building our approach to searching and retrieving. The task of the Quora duplicate question pair is to determine whether two pairs of questions have the same meaning. The data contains a pair of questions and a ground truth label, marked by a human expert, mentioning whether the pairs of questions are duplicates. The ground truth data provided also mentions that these labels are subjective, meaning that not all human experts might agree on whether the pair of questions is similar. Hence, the data should be taken as informed, and not 100% accurate, due to the inherent subjectivity in ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required