Skip to Main Content
Python 3 Text Processing with NLTK 3 Cookbook - Second Edition
book

Python 3 Text Processing with NLTK 3 Cookbook - Second Edition

by Jacob Perkins
August 2014
Beginner to intermediate content levelBeginner to intermediate
304 pages
7h 10m
English
Packt Publishing
Content preview from Python 3 Text Processing with NLTK 3 Cookbook - Second Edition

Flattening a deep tree

Some of the included corpora contain parsed sentences, which are often deep trees of nested phrases. Unfortunately, these trees are too deep to use for training a chunker, since IOB tag parsing is not designed for nested chunks. To make these trees usable for chunker training, we must flatten them.

Getting ready

We're going to use the first parsed sentence of the treebank corpus as our example. Here's a diagram showing how deeply nested this tree is:

Getting ready

You may notice that the part-of-speech tags are part of the tree structure instead of being included with the word. This will be handled later using the Tree.pos() method, which ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Python Machine Learning - Third Edition

Python Machine Learning - Third Edition

Sebastian Raschka, Vahid Mirjalili
Python Cookbook, 3rd Edition

Python Cookbook, 3rd Edition

David Beazley, Brian K. Jones

Publisher Resources

ISBN: 9781782167853Supplemental Content