book

Natural Language Understanding with Python

Name: Natural Language Understanding with Python
Author: Deborah A. Dahl
ISBN: 9781804613429

by Deborah A. Dahl

June 2023

Intermediate to advanced

326 pages

English

Packt Publishing

Read now

Unlock full access

Natural Language Understanding with Python
ContributorsAbout the authorAbout the reviewersAcknowledgment
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare your thoughtsDownload a free PDF copy of this book
Part 1: Getting Started with Natural Language Understanding Technology
Chapter 1: Natural Language Understanding, Related Technologies, and Natural Language Applications
Understanding the basics of natural languageGlobal considerations – languages, encodings, and translationsThe relationship between conversational AI and NLPExploring interactive applications – chatbots and voice assistantsGeneric voice assistantsEnterprise assistantsTranslationEducationExploring non-interactive applicationsClassificationSentiment analysisSpam and phishing detectionFake news detectionDocument retrievalAnalyticsInformation extractionTranslationSummarization, authorship, correcting grammar, and other applicationsA summary of the types of applicationsA look ahead – Python for NLPSummary
Chapter 2: Identifying Practical Natural Language Understanding Problems
Identifying problems that are the appropriate level of difficulty for the technologyLooking at difficult applications of NLULooking at applications that don’t need NLPTraining dataApplication dataTaking development costs into accountTaking maintenance costs into accountA flowchart for deciding on NLU applicationsSummary
Part 2:Developing and Testing Natural Language Understanding Systems
Chapter 3: Approaches to Natural Language Understanding – Rule-Based Systems, Machine Learning, and Deep Learning
Rule-based approachesWords and lexiconsPart-of-speech taggingGrammarParsingSemantic analysisPragmatic analysisPipelinesTraditional machine learning approachesRepresenting documentsClassificationDeep learning approachesPre-trained modelsConsiderations for selecting technologiesSummary
Chapter 4: Selecting Libraries and Tools for Natural Language Understanding
Technical requirementsInstalling PythonDeveloping software – JupyterLab and GitHubJupyterLabGitHubExploring the librariesUsing NLTKUsing spaCyUsing KerasLearning about other NLP librariesChoosing among NLP librariesLearning about other packages useful for NLPLooking at an exampleSetting up JupyterLabProcessing one sentenceLooking at corpus propertiesSummary
Chapter 5: Natural Language Data – Finding and Preparing Data
Finding sources of data and annotating itFinding data for your own applicationFinding data for a research projectMetadataGenerally available corporaEnsuring privacy and observing ethical considerationsEnsuring the privacy of training dataEnsuring the privacy of runtime dataTreating human subjects ethicallyTreating crowdworkers ethicallyPreprocessing dataRemoving non-textRegularizing textSpelling correctionApplication-specific types of preprocessingSubstituting class labels for words and numbersRedactionDomain-specific stopwordsRemove HTML markupData imbalanceUsing text preprocessing pipelinesChoosing among preprocessing techniquesSummary
Chapter 6: Exploring and Visualizing Data
Why visualize?Text document dataset – Sentence Polarity DatasetData explorationFrequency distributionsMeasuring the similarities among documentsGeneral considerations for developing visualizationsUsing information from visualization to make decisions about processingSummary

Chapter 7: Selecting Approaches and Representing Data
Selecting NLP approachesFitting the approach to the taskStarting with the dataConsidering computational efficiencyInitial studiesRepresenting language for NLP applicationsSymbolic representationsRepresenting language numerically with vectorsUnderstanding vectors for document representationRepresenting words with context-independent vectorsWord2VecRepresenting words with context-dependent vectorsSummary
Chapter 8: Rule-Based Techniques
Rule-based techniquesWhy use rules?Exploring regular expressionsRecognizing, parsing, and replacing strings with regular expressionsGeneral tips for using regular expressionsWord-level analysisLemmatizationOntologiesSentence-level analysisSyntactic analysisSemantic analysis and slot fillingSummary
Chapter 9: Machine Learning Part 1 – Statistical Machine Learning
A quick overview of evaluationRepresenting documents with TF-IDF and classifying with Naïve BayesSummary of TF-IDFClassifying texts with Naïve BayesTF-IDF/Bayes classification exampleClassifying documents with Support Vector Machines (SVMs)Slot-filling with CRFsRepresenting slot-tagged dataSummary
Chapter 10: Machine Learning Part 2 – Neural Networks and Deep Learning Techniques
Basics of NNsExample – MLP for classificationHyperparameters and tuningMoving beyond MLPs – RNNsLooking at another approach – CNNsSummary
Chapter 11: Machine Learning Part 3 – Transformers and Large Language Models
Technical requirementsOverview of transformers and LLMsIntroducing attentionApplying attention in transformersLeveraging existing data – LLMs or pre-trained modelsBERT and its variantsUsing BERT – a classification exampleInstalling the dataSplitting the data into training, validation, and testing setsLoading the BERT modelDefining the model for fine-tuningDefining the loss function and metricsDefining the optimizer and the number of epochsCompiling the modelTraining the modelPlotting the training processEvaluating the model on the test dataSaving the model for inferenceCloud-based LLMsChatGPTApplying GPT-3Summary
Chapter 12: Applying Unsupervised Learning Approaches
What is unsupervised learning?Topic modeling using clustering techniques and label derivationGrouping semantically similar documentsApplying BERTopic to 20 newsgroupsAfter clustering and topic labelingMaking the most of data with weak supervisionSummary
Chapter 13: How Well Does It Work? – Evaluation
Why evaluate an NLU system?Evaluation paradigmsComparing system results on standard metricsEvaluating language outputLeaving out part of a system – ablationShared tasksData partitioningEvaluation metricsAccuracy and error ratePrecision, recall, and F1The receiver operating characteristic and area under the curveConfusion matrixUser testingStatistical significance of differencesComparing three text classification methodsA small transformer systemTF-IDF evaluationA larger BERT modelSummary
Part 3: Systems in Action – Applying Natural Language Understanding at Scale
Chapter 14: What to Do If the System Isn’t Working
Technical requirementsFiguring out that a system isn’t workingInitial developmentFixing accuracy problemsChanging dataRestructuring an applicationMoving on to deploymentProblems after deploymentSummary
Chapter 15: Summary and Looking to the Future
Overview of the bookPotential for improvement – better accuracy and faster trainingBetter accuracyFaster trainingOther areas for improvementApplications that are beyond the current state of the artProcessing very long documentsUnderstanding and creating videosInterpreting and generating sign languagesWriting compelling fictionFuture directions in NLU technology and researchQuickly extending NLU technologies to new languagesReal-time speech-to-speech translationMultimodal interactionDetecting and correcting biasSummaryFurther reading
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a free PDF copy of this book

Content preview from Natural Language Understanding with Python

3 Approaches to Natural Language Understanding – Rule-Based Systems, Machine Learning, and Deep Learning

This chapter will review the most common approaches to natural language understanding (NLU) and discuss both the benefits and drawbacks of each approach, including rule-based techniques, statistical techniques, and deep learning. It will also discuss popular pre-trained models such as Bidirectional Encoder Representations from Transformers (BERT) and its variants. We will learn that NLU is not a single technology; it includes a range of techniques, which are applicable to different goals.

In this chapter, we cover the following main topics:

Rule-based approaches
Traditional machine-learning approaches
Deep learning approaches
Pre-trained ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Natural Language Processing with Python and spaCy

Publisher Resources

ISBN: 9781804613429

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Natural Language Understanding with Python

by Deborah A. Dahl

3

Approaches to Natural Language Understanding – Rule-Based Systems, Machine Learning, and Deep Learning

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.