book

Hands-On Python Natural Language Processing

Name: Hands-On Python Natural Language Processing
ISBN: 9781838989590

by Aman Kedia, Mayank Rasu

June 2020

Beginner to intermediate

316 pages

6h 44m

English

Packt Publishing

Read now

Unlock full access

Title Page
Copyright and Credits
Hands-On Python Natural Language Processing
About Packt
Why subscribe?
Contributors
About the authorsAbout the reviewersPackt is searching for authors like you
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Section 1: Introduction
Understanding the Basics of NLP
Programming languages versus natural languagesUnderstanding NLPWhy should I learn NLP?Current applications of NLPChatbotsSentiment analysisMachine translationNamed-entity recognitionFuture applications of NLPSummary
NLP Using Python
Technical requirementsUnderstanding Python with NLPPython's utility in NLPImportant Python librariesNLTKNLTK corporaText processingPart of speech taggingTextblobSentiment analysisMachine translationPart of speech taggingVADERWeb scraping libraries and methodologyOverview of Jupyter NotebookSummary
Section 2: Natural Language Representation and Mathematics
Building Your NLP Vocabulary
Technical requirementsLexiconsPhonemes, graphemes, and morphemesTokenizationIssues with tokenizationDifferent types of tokenizersRegular expressionsRegular expressions-based tokenizersTreebank tokenizerTweetTokenizerUnderstanding word normalizationStemmingOver-stemming and under-stemmingLemmatizationWordNet lemmatizerSpacy lemmatizerStopword removalCase foldingN-gramsTaking care of HTML tagsHow does all this fit into my NLP pipeline?Summary

Transforming Text into Data Structures
Technical requirementsUnderstanding vectors and matricesVectorsMatricesExploring the Bag-of-Words architectureUnderstanding a basic CountVectorizerOut-of-the-box features offered by CountVectorizerPrebuilt dictionary and support for n-gramsmax_featuresMin_df and Max_df thresholdsLimitations of the BoW representationTF-IDF vectorsBuilding a basic TF-IDF vectorizerN-grams and maximum features in the TF-IDF vectorizer Limitations of the TF-IDF vectorizer's representationDistance/similarity calculation between document vectorsCosine similaritySolving Cosine mathCosine similarity on vectors developed using CountVectorizerCosine similarity on vectors developed using TfIdfVectorizers toolOne-hot vectorizationBuilding a basic chatbotSummary
Word Embeddings and Distance Measurements for Text
Technical requirementsUnderstanding word embeddingsDemystifying Word2vecSupervised and unsupervised learningWord2vec – supervised or unsupervised?Pretrained Word2vec Exploring the pretrained Word2vec model using gensimThe Word2vec architectureThe Skip-gram methodHow do you define target and context words?Exploring the components of a Skip-gram modelInput vectorEmbedding matrixContext matrixOutput vectorSoftmaxLoss calculation and backpropagationInferenceThe CBOW methodComputational limitations of the methods discussed and how to overcome themSubsamplingNegative samplingHow to select negative samplesTraining a Word2vec model Building a basic Word2vec modelModifying the min_count parameter Playing with the vector sizeOther important configurable parametersLimitations of Word2vecApplications of the Word2vec model Word mover’s distanceSummary
Exploring Sentence-, Document-, and Character-Level Embeddings
Technical requirementsVenturing into Doc2VecBuilding a Doc2Vec modelChanging vector size and min_count The dm parameter for switching between modeling approachesThe dm_concat parameterThe dm_mean parameterWindow sizeLearning rateExploring fastText Building a fastText modelBuilding a spelling corrector/word suggestion module using fastTextfastText and document distancesUnderstanding Sent2Vec and the Universal Sentence Encoder</span>Sent2VecThe Universal Sentence EncoderSummary
Section 3: NLP and Learning
Identifying Patterns in Text Using Machine Learning
Technical requirementsIntroduction to MLData preprocessingNaN valuesLabel encoding and one-hot encodingData standardizationMin-max standardizationZ-score standardizationThe Naive Bayes algorithmBuilding a sentiment analyzer using the Naive Bayes algorithmThe SVM algorithmBuilding a sentiment analyzer using SVMProductionizing a trained sentiment analyzerSummary
From Human Neurons to Artificial Neurons for Understanding Text
Technical requirementsExploring the biology behind neural networksNeuronsActivation functionsSigmoidTanh activationRectified linear unitLayers in an ANNHow does a neural network learn?How does the network get better at making predictions?Understanding regularizationDropoutLet's talk KerasBuilding a question classifier using neural networksSummary
Applying Convolutions to Text
Technical requirementsWhat is a CNN?Understanding convolutionsLet's pad our dataUnderstanding strides in a CNNWhat is pooling?The fully connected layerDetecting sarcasm in text using CNNsLoading the libraries and the datasetPerforming basic data analysis and preprocessing our dataLoading the Word2Vec model and vectorizing our dataSplitting our dataset into train and test setsBuilding the modelEvaluating and saving our modelSummary
Capturing Temporal Relationships in Text
Technical requirementsBaby steps toward understanding RNNsForward propagation in an RNNBackpropagation through time in an RNNVanishing and exploding gradientsArchitectural forms of RNNsDifferent flavors of RNNCarrying relationships both ways using bidirectional RNNsGoing deep with RNNsGiving memory to our networks – LSTMsUnderstanding an LSTM cellForget gateInput gateOutput gateBackpropagation through time in LSTMsBuilding a text generator using LSTMsExploring memory-based variants of the RNN architectureGRUsStacked LSTMsSummary
State of the Art in NLP
Technical requirementsSeq2Seq modelingEncodersDecodersThe training phaseThe inference phaseTranslating between languages using Seq2Seq modeling Let's pay some attentionTransformers Understanding the architecture of TransformersEncoders DecodersSelf-attentionHow does self-attention work mathematically?A small note on masked self-attentionFeedforward neural networksResiduals and layer normalizationPositional embeddingsHow the decoder worksThe linear layer and the softmax functionTransformer model summaryBERT The BERT architectureThe BERT model input and outputHow did BERT the pre-training happen?The masked language modelNext-sentence prediction BERT fine-tuningSummary
Other Books You May Enjoy
Leave a review - let other readers know what you think

Content preview from Hands-On Python Natural Language Processing

Capturing Temporal Relationships in Text

In the previous chapters, we saw how we could leverage Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) to mine patterns in text and apply them to various tasks such as classifying questions and sarcasm detection in news headlines. With ANNs, we primarily saw that inputs are independent of one another. With CNNs, we went one step further and tried to capture spatial relationships in the inputs by trying to extract patterns across a set of tokens together. However, our scope was limited to only a few tokens in the vicinity.

Sentences are essentially sequences of words, and the contextual meaning of a particular word in a sentence may not be derived solely from the immediately ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781838989590

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Hands-On Python Natural Language Processing

by Aman Kedia, Mayank Rasu

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.