book

Natural Language Processing with Python

by Steven Bird, Ewan Klein, Edward Loper

June 2009

Beginner to intermediate

504 pages

16h 27m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Audience

Computing with Language: Texts and WordsGetting Started with PythonGetting Started with NLTKSearching TextCounting Vocabulary
ListsIndexing ListsVariablesStrings
Frequency DistributionsFine-Grained Selection of WordsCollocations and BigramsCounting Other Things
ConditionalsOperating on Every ElementNested Code BlocksLooping with Conditions
Word Sense DisambiguationPronoun ResolutionGenerating Language OutputMachine TranslationSpoken Dialogue SystemsTextual EntailmentLimitations of NLP
Accessing Text CorporaGutenberg CorpusWeb and Chat TextBrown CorpusReuters CorpusInaugural Address CorpusAnnotated Text CorporaCorpora in Other LanguagesText Corpus StructureLoading Your Own Corpus
Conditions and EventsCounting Words by GenrePlotting and Tabulating DistributionsGenerating Random Text with Bigrams
Creating Programs with a Text EditorFunctionsModules
Wordlist CorporaA Pronouncing DictionaryComparative WordlistsShoebox and Toolbox Lexicons
Senses and SynonymsThe WordNet HierarchyMore Lexical RelationsSemantic Similarity
Accessing Text from the Web and from DiskElectronic BooksDealing with HTMLProcessing Search Engine ResultsProcessing RSS FeedsReading Local FilesExtracting Text from PDF, MSWord, and Other Binary FormatsCapturing User InputThe NLP Pipeline
Basic Operations with StringsPrinting StringsAccessing Individual CharactersAccessing SubstringsMore Operations on StringsThe Difference Between Lists and Strings
What Is Unicode?Extracting Encoded Text from FilesUsing Your Local Encoding in Python
Using Basic MetacharactersRanges and Closures
Extracting Word PiecesDoing More with Word PiecesFinding Word StemsSearching Tokenized Text
StemmersLemmatization
Simple Approaches to TokenizationNLTK’s Regular Expression TokenizerFurther Issues with Tokenization
Sentence SegmentationWord Segmentation
From Lists to StringsStrings and FormatsLining Things UpWriting Results to a FileText Wrapping
Back to the BasicsAssignmentEqualityConditionals
Operating on Sequence TypesCombining Different Sequence TypesGenerator Expressions
Python Coding StyleProcedural Versus Declarative StyleSome Legitimate Uses for Counters
Function Inputs and OutputsParameter PassingVariable ScopeChecking Parameter TypesFunctional DecompositionDocumenting Functions
Functions As ArgumentsAccumulative FunctionsHigher-Order FunctionsNamed Arguments
Structure of a Python ModuleMultimodule ProgramsSources of ErrorDebugging TechniquesDefensive Programming
RecursionSpace-Time Trade-offsDynamic Programming
MatplotlibNetworkXcsvNumPyOther Python Libraries
Using a Tagger
Representing Tagged TokensReading Tagged CorporaA Simplified Part-of-Speech TagsetNounsVerbsAdjectives and AdverbsUnsimplified TagsExploring Tagged Corpora
Indexing Lists Versus DictionariesDictionaries in PythonDefining DictionariesDefault DictionariesIncrementally Updating a DictionaryComplex Keys and ValuesInverting a Dictionary
The Default TaggerThe Regular Expression TaggerThe Lookup TaggerEvaluation
Unigram TaggingSeparating the Training and Testing DataGeneral N-Gram TaggingCombining TaggersTagging Unknown WordsStoring TaggersPerformance LimitationsTagging Across Sentence Boundaries
Morphological CluesSyntactic CluesSemantic CluesNew WordsMorphology in Part-of-Speech Tagsets
Supervised ClassificationGender IdentificationChoosing the Right FeaturesDocument ClassificationPart-of-Speech TaggingExploiting ContextSequence ClassificationOther Methods for Sequence Classification
Sentence SegmentationIdentifying Dialogue Act TypesRecognizing Textual EntailmentScaling Up to Large Datasets
The Test SetAccuracyPrecision and RecallConfusion MatricesCross-Validation
Entropy and Information Gain
Underlying Probabilistic ModelZero Counts and SmoothingNon-Binary FeaturesThe Naivete of IndependenceThe Cause of Double-Counting
The Maximum Entropy ModelMaximizing EntropyGenerative Versus Conditional Classifiers
What Do Models Tell Us?
Information ExtractionInformation Extraction Architecture
Noun Phrase ChunkingTag PatternsChunking with Regular ExpressionsExploring Text CorporaChinkingRepresenting Chunks: Tags Versus Trees
Reading IOB Format and the CoNLL-2000 Chunking CorpusSimple Evaluation and BaselinesTraining Classifier-Based Chunkers
Building Nested Structure with Cascaded ChunkersTreesTree Traversal
Some Grammatical DilemmasLinguistic Data and Unlimited PossibilitiesUbiquitous Ambiguity
Beyond n-grams
A Simple GrammarWriting Your Own GrammarsRecursion in Syntactic Structure
Recursive Descent ParsingShift-Reduce ParsingThe Left-Corner ParserWell-Formed Substring Tables
Valency and the LexiconScaling Up
Treebanks and GrammarsPernicious AmbiguityWeighted Grammar
Grammatical FeaturesSyntactic AgreementUsing Attributes and ConstraintsTerminology
Subsumption and Unification
SubcategorizationHeads RevisitedAuxiliary Verbs and InversionUnbounded Dependency ConstructionsCase and Gender in German
Natural Language UnderstandingQuerying a DatabaseNatural Language, Semantics, and Logic
SyntaxFirst-Order Theorem ProvingSummarizing the Language of First-Order LogicTruth in ModelIndividual Variables and AssignmentsQuantificationQuantifier Scope AmbiguityModel Building
Compositional Semantics in Feature-Based GrammarThe λ-CalculusQuantified NPsTransitive VerbsQuantifier Ambiguity Revisited
Discourse Representation TheoryDiscourse Processing
Corpus Structure: A Case StudyThe Structure of TIMITNotable Design FeaturesFundamental Data Types
Three Corpus Creation ScenariosQuality ControlCuration Versus Evolution
Obtaining Data from the WebObtaining Data from Word Processor FilesObtaining Data from Spreadsheets and DatabasesConverting Data FormatsDeciding Which Layers of Annotation to IncludeStandards and ToolsSpecial Considerations When Working with Endangered Languages
Using XML for Linguistic StructuresThe Role of XMLThe ElementTree InterfaceUsing ElementTree for Accessing Toolbox DataFormatting Entries
Adding a Field to Each EntryValidating a Toolbox Lexicon
What Is Metadata?OLAC: Open Language Archives Community
Language Processing Versus Symbol Processing

Content preview from Natural Language Processing with Python

Royalties

Royalties from the sale of this book are being used to support the development of the Natural Language Toolkit.

Figure 1. Edward Loper, Ewan Klein, and Steven Bird, Stanford, July 2007

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Natural Language Processing with Python and spaCy

Yuli Vasiliev

Hands-On Natural Language Processing with Python

Rajesh Arumugam, Rajalingappaa Shanmugamani

Natural Language Processing with PyTorch

Delip Rao, Brian McMahan

Python Natural Language Processing Cookbook

Zhenya Antić

Publisher Resources

ISBN: 9780596803346Errata Page