August 2018
Intermediate to advanced
522 pages
12h 45m
English
In many cases, it's useful to split large texts into sentences, which are normally delimited by a full stop or another equivalent mark. As every language has its own orthographic rules, NLTK offers the sent_tokenize() method which accepts a language (the default is English) and splits the text according to the specific rules. In the following example, we show the usage of this function with different languages:
from nltk.tokenize import sent_tokenizegeneric_text = 'Lorem ipsum dolor sit amet, amet minim temporibus in sit. Vel ne impedit consequat intellegebat.'print(sent_tokenize(generic_text))['Lorem ipsum dolor sit amet, amet minim temporibus in sit.', 'Vel ne impedit consequat intellegebat.']english_text = 'Where is ...
Read now
Unlock full access