O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Dialog datasets

Dialog tasks are generally divided into two broad categories: open-ended conversation (also known as chit-chat) and goal-oriented systems.

Open-ended dialog systems generally deal with conversing on unrestricted subjects and are trained using large scale corpuses from Twitter conversations, reddit replies, or similar forum posts. Since most open-ended tasks require the generation of responses, most models use the seq2seq framework, similar to machine translation or text summarization, and are evaluated using a combination of translation metrics (such as BLEU score) and human evaluation.

The key challenges involved in building these neural conversational models besides language modelling and generation are the lack of consistent ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required