book

Natural Language Understanding with Python

by Deborah A. Dahl

June 2023

Intermediate to advanced

326 pages

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewersAcknowledgment
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare your thoughtsDownload a free PDF copy of this book
Understanding the basics of natural languageGlobal considerations – languages, encodings, and translationsThe relationship between conversational AI and NLPExploring interactive applications – chatbots and voice assistantsGeneric voice assistantsEnterprise assistantsTranslationEducationExploring non-interactive applicationsClassificationSentiment analysisSpam and phishing detectionFake news detectionDocument retrievalAnalyticsInformation extractionTranslationSummarization, authorship, correcting grammar, and other applicationsA summary of the types of applicationsA look ahead – Python for NLPSummary
Identifying problems that are the appropriate level of difficulty for the technologyLooking at difficult applications of NLULooking at applications that don’t need NLPTraining dataApplication dataTaking development costs into accountTaking maintenance costs into accountA flowchart for deciding on NLU applicationsSummary
Rule-based approachesWords and lexiconsPart-of-speech taggingGrammarParsingSemantic analysisPragmatic analysisPipelinesTraditional machine learning approachesRepresenting documentsClassificationDeep learning approachesPre-trained modelsConsiderations for selecting technologiesSummary
Technical requirementsInstalling PythonDeveloping software – JupyterLab and GitHubJupyterLabGitHubExploring the librariesUsing NLTKUsing spaCyUsing KerasLearning about other NLP librariesChoosing among NLP librariesLearning about other packages useful for NLPLooking at an exampleSetting up JupyterLabProcessing one sentenceLooking at corpus propertiesSummary
Finding sources of data and annotating itFinding data for your own applicationFinding data for a research projectMetadataGenerally available corporaEnsuring privacy and observing ethical considerationsEnsuring the privacy of training dataEnsuring the privacy of runtime dataTreating human subjects ethicallyTreating crowdworkers ethicallyPreprocessing dataRemoving non-textRegularizing textSpelling correctionApplication-specific types of preprocessingSubstituting class labels for words and numbersRedactionDomain-specific stopwordsRemove HTML markupData imbalanceUsing text preprocessing pipelinesChoosing among preprocessing techniquesSummary
Why visualize?Text document dataset – Sentence Polarity DatasetData explorationFrequency distributionsMeasuring the similarities among documentsGeneral considerations for developing visualizationsUsing information from visualization to make decisions about processingSummary

Selecting NLP approachesFitting the approach to the taskStarting with the dataConsidering computational efficiencyInitial studiesRepresenting language for NLP applicationsSymbolic representationsRepresenting language numerically with vectorsUnderstanding vectors for document representationRepresenting words with context-independent vectorsWord2VecRepresenting words with context-dependent vectorsSummary
Rule-based techniquesWhy use rules?Exploring regular expressionsRecognizing, parsing, and replacing strings with regular expressionsGeneral tips for using regular expressionsWord-level analysisLemmatizationOntologiesSentence-level analysisSyntactic analysisSemantic analysis and slot fillingSummary
A quick overview of evaluationRepresenting documents with TF-IDF and classifying with Naïve BayesSummary of TF-IDFClassifying texts with Naïve BayesTF-IDF/Bayes classification exampleClassifying documents with Support Vector Machines (SVMs)Slot-filling with CRFsRepresenting slot-tagged dataSummary
Basics of NNsExample – MLP for classificationHyperparameters and tuningMoving beyond MLPs – RNNsLooking at another approach – CNNsSummary
Technical requirementsOverview of transformers and LLMsIntroducing attentionApplying attention in transformersLeveraging existing data – LLMs or pre-trained modelsBERT and its variantsUsing BERT – a classification exampleInstalling the dataSplitting the data into training, validation, and testing setsLoading the BERT modelDefining the model for fine-tuningDefining the loss function and metricsDefining the optimizer and the number of epochsCompiling the modelTraining the modelPlotting the training processEvaluating the model on the test dataSaving the model for inferenceCloud-based LLMsChatGPTApplying GPT-3Summary
What is unsupervised learning?Topic modeling using clustering techniques and label derivationGrouping semantically similar documentsApplying BERTopic to 20 newsgroupsAfter clustering and topic labelingMaking the most of data with weak supervisionSummary
Why evaluate an NLU system?Evaluation paradigmsComparing system results on standard metricsEvaluating language outputLeaving out part of a system – ablationShared tasksData partitioningEvaluation metricsAccuracy and error ratePrecision, recall, and F1The receiver operating characteristic and area under the curveConfusion matrixUser testingStatistical significance of differencesComparing three text classification methodsA small transformer systemTF-IDF evaluationA larger BERT modelSummary
Technical requirementsFiguring out that a system isn’t workingInitial developmentFixing accuracy problemsChanging dataRestructuring an applicationMoving on to deploymentProblems after deploymentSummary
Overview of the bookPotential for improvement – better accuracy and faster trainingBetter accuracyFaster trainingOther areas for improvementApplications that are beyond the current state of the artProcessing very long documentsUnderstanding and creating videosInterpreting and generating sign languagesWriting compelling fictionFuture directions in NLU technology and researchQuickly extending NLU technologies to new languagesReal-time speech-to-speech translationMultimodal interactionDetecting and correcting biasSummaryFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a free PDF copy of this book

Overview

"Natural Language Understanding with Python" equips readers with the skills to develop advanced natural language processing (NLP) applications. You'll learn essential concepts like text analysis, deep learning for NLP, and using large language models effectively. By implementing practical Python examples, you will harness the power of NLP for building chatbots, sentiment analysis systems, and more.

What this Book will help me do

Understand the fundamentals of natural language understanding (NLU) and its real-world applications.
Learn how to apply libraries and techniques to process and analyze textual data effectively.
Develop practical NLP systems, such as sentiment analysis tools and chatbots.
Master the use of large language models, including ChatGPT, for NLU tasks.
Gain expertise in deploying and maintaining NLU systems in production environments.

Author(s)

Deborah A. Dahl is a seasoned computational linguist with decades of experience in natural language processing and understanding. She has worked on numerous applications, ranging from conversational AI to advanced text analysis systems. Deborah shares her insights through a clear, hands-on approach that makes complex topics accessible and engaging for learners.

Who is it for?

This book is designed for Python developers, data scientists, and NLP practitioners who want to delve into natural language understanding. It is also an excellent resource for students and researchers aiming to apply NLP techniques in practical situations. Readers should have basic programming skills and an interest in text processing and NLP applications.