book

Mastering Text Analytics : A Hands-on Guide to NLP Using Python

Name: Mastering Text Analytics : A Hands-on Guide to NLP Using Python
ISBN: 9798868815829

by Shailendra Kadre, Shailesh Kadre, Subhendu Dey

August 2025

Intermediate to advanced

494 pages

9h 46m

English

Apress

Read now

Unlock full access

Mastering Text Analytics
Introduction
Acknowledgments
Table of Contents
About the Authors
About the Technical Reviewer
1. Natural Language Processing: An Introduction
1.1 Why You Should Read This Chapter1.2 Introduction to Natural Language Processing1.2.1 NLP Techniques1.2.2 Text Classification1.2.3 Clustering1.2.4 Collocation1.2.5 Concordance (Computing Word Frequencies)1.2.6 Text Extraction1.2.7 Stop Word Removal1.2.8 Word Sense Disambiguation (WSD)1.3 Challenges in NLP1.4 Applications of NLP1.4.1 Banking and Financial Industry (BFSI)1.4.2 Healthcare1.4.3 Legal1.4.4 Automate Routine Business Processes1.4.5 Improve Search1.4.6 Search Engine Optimization1.4.7 Machine Translation: Translating Languages Automatically Using NLP1.4.8 Text Summarization: Condensing Large Text Information1.5 Recent Trends and Future Directions1.5.1 Transfer Learning: Advantages of Pre-Training1.5.2 LLMs: Human-like Interactions1.5.3 Cross-Lingual and Multiple-Lingual Models: Handling Multiple Languages1.5.4 Explainable NLP: Reason Model Results1.5.5 Emergent Areas in NLP1.6 Short Business Case 1: Application of NLP in Medical Text Analysis1.6.1 Streamlining Clinical Research1.6.2 Improving Administrative Efficiency1.6.3 Enhancing Data Accessibility1.6.4 Facilitating Predictive Analytics1.6.5 Business Case Conclusion1.7 Short Business Case 2: NLP in Customer Service, Enhancing Customer Interaction1.7.1 The Challenge1.7.2 Implementation of NLP1.7.3 Results and Benefits1.7.4 Conclusion1.8 Example with Python1.8.1 NLP Tutorial1.8.2 YouTube Comments Spam Detection Using PythonThe DataBuilding VectorsA Second ExampleCounts for the First Five Entries1.9 Recap of Key Concepts1.10 Conclusion1.11 Exercises
2. Collecting and Extracting the Data for NLP Projects
2.1 Why You Should Learn NLP2.2 Where to Find the Data (Text Corpora) for NLP Projects2.2.1 Accessing Some Popular NLTK Text Corpora2.2.2 Accessing Gutenberg Corpus2.2.3 Accessing Reuters Corpus2.2.4 Accessing Brown Corpus2.2.5 Accessing Gutenberg Corpus2.2.6 Accessing Web and Chat Text2.2.7 Accessing NLTK Lexical Resources2.3 Extracting Data from Word Files2.3.1 Extracting Data from MS Word Files2.4 Extracting Data from HTML2.4.1 Extracting Data from HTML Documents2.5 Extracting Data from JSON2.5.1 Extracting Data from JSON Files2.6 Extracting Data from PDFs2.6.1 Extracting Data from JSON Files2.7 Web Scraping2.7.1 Web Scraping in NLP Tasks2.8 Recap of Key Concepts2.9 Practice Exercises
3. NLP Data Preprocessing Tasks Involving Strings and Python Regular Expressions
3.1 Why You Should Read This Chapter3.2 Python for Language Processing3.2.1 A Text Analytics Project Life Cycle (Generic NLP Pipeline)3.2.2 String Handling in PythonCreating a StringAccessing Different Characters in a StringMore Basic String Operations3.2.3 Introducing Regular Expressions for Text Processing3.3 Real-life (NLP) Applications of RegEx3.3.1 Validate Formatting of Phone Numbers3.3.2 Search and Replace3.3.3 Date Formatting3.3.4 Word Counting3.3.5 Log File Analysis3.3.6 String Cleaning3.3.7 Tokenization3.3.8 Text Normalization3.4 Chapter Recap3.5 Exercises
4. NLP Data Preprocessing Tasks with NLTK
4.1 Why You Should Read This Chapter4.2 NLP Data Preprocessing Tasks4.2.1 Tokenization Using NLTK4.2.2 Stop Word Removal4.2.3 Stemming Using NLTK4.2.4 Lemmatization4.2.5 Sentence Segmentation4.2.6 Word Frequency Distribution4.2.7 Antonym and Synonym Detection Using NLTK4.2.8 Word Similarity Calculation4.2.9 Word Sense Disambiguation Using NLTK4.2.10 Keyword Extraction with NLTK4.3 NLP Tasks Beyond Preprocessing4.3.1 Part-of-Speech Tagging4.3.2 Named Entity Recognition with NLTK4.4 Some Useful Functionalities Outside NLTK4.4.1 Word Cloud Generation4.4.2 Language Translation4.4.3 Text Summarization4.5 Chapter Recap4.6 Exercises

5. Lexical Analysis
5.1 Why You Should Read This Chapter5.2 Morphological Analysis Revisited5.3 Tokenization Revisited5.3.1 Code DemosWhitespace TokenizationDictionary-Based TokenizationRule-based TokenizationPunctuation-Based TokenizerTweet TokenizerMWE TokenizerBag-of-Words Model5.4 Stemming Revisited5.4.1 Code Demo for Multiple StemmersSnowball or Porter2 StemmerLancaster Stemmer5.5 Lemmatization Revisited5.5.1 Code Demo for Multiple LemmatizersWordNet Lemmatizer (NLTK)The spaCy LemmatizerTextBlob Lemmatizer5.6 A Complete Lexical Analysis with Data Preprocessing5.6.1 YouTube Comments Spam Detection5.6.2 Convert the Text to Lowercase5.6.3 Remove All Unwanted Punctuation5.6.4 Remove Stop Words5.6.5 Remove Frequent Words5.6.6 Remove Rare Words5.6.7 Stemming5.6.8 Lemmatization5.6.9 Removal of Emojis5.6.10 Removal of EmoticonsConverting Emoticons to WordsConversion of Emojis to Words5.6.11 Removal of URLs5.6.12 Removal of HTML tags5.6.13 Chat Words Conversion5.6.14 Spelling Corrections5.6.15 Apply Machine Learning Model5.7 Chapter Recap
6. Syntactic and Semantic Techniques in NLP
6.1 Why You Should Read This Chapter6.2 An Overview of Structural Techniques6.2.1 Part-of-Speech Tagging6.2.2 POS Tagging TutorialsPOS Tagging with NLTKPOS Tagging with spaCy6.2.3 Parsing TechniquesDependency ParsingConstituency Parsing6.2.4 Hands-on Example Covering for ParsingDependency Parsing Code Demo6.2.5 Challenges in Syntactic Analysis6.3 Introduction to Semantics6.3.1 Chunking in NLP6.3.2 Named Entity Recognition6.3.3 WSD Revisited6.3.4 Term-Document Matrix (Co-Occurrence Matrix)6.3.5 Term Frequency–Inverse Document Frequency6.3.6 Coding Tutorials for This SectionChunkingNamed Entity RecognitionWord Sense DisambiguationTerm-Document MatrixTF–IDF6.4 Lexical Semantics6.4.1 Synonyms6.4.2 Antonyms6.4.3 Homophones6.4.4 Homographs6.4.5 Polysemy6.4.6 Hyponyms6.4.7 Building Lexical Semantic Models with Code ExamplesTutorial on Synonyms, Antonyms, Homophones, Homographs, Polysemy, and HyponymsTokenize the ArticleSynonyms DetectionAntonyms DetectionHomophones Detection (Alternate Approach)Homographs DetectionPolysemy DetectionHyponyms of a Given WordCategorize the Article Based on Hyponyms of Given Categories6.4.8 Integration into Text Processing PipelinesDemonstrating Text Preprocessing Pipelines Using POS Tagging, NER, and Polysemy6.5 Chapter Recap
7. Advanced Pragmatic Techniques and Specialized Topics in NLP
7.1 Why You Should Read This Chapter7.2 Overview of Pragmatic Analysis Techniques7.3 Discourse Integration in NLP7.3.1 Techniques for Discourse Integration7.3.2 Demonstrating Techniques for Discourse IntegrationCoreference Resolution (Code Demo)Rhetorical Structure Theory (Code Demo)Discourse Parsing (Code Demo)Entity Tracking (Code Demo)Lexical Chains (Code Demo)Topic Modeling (Code Demo)Cohesion Analysis (Code Demo)Temporal Relation Identification (Code Demo)7.4 Distributional Semantics and Word Embeddings7.4.1 Latent Semantic AnalysisCode Demo of LSA7.4.2 Popular Word EmbeddingsTraditional Word EmbeddingsWord2VecContinuous Bag of Words ModelSkip-gram ModelCBOW and Skip-gram Code Demonstration Using Word2VecGloVe: Global Vectors for Word RepresentationAdvances like BERT and GPT7.4.3 Case Studies and Applications7.5 Chapter Recap
8. Transformers, Generative AI, and LangChain
8.1 Introduction to Transformers in NLP8.1.1 Evolution of NLP: From Traditional Models to Transformers8.1.2 Overview of BERT by Google8.1.3 Code to Demonstrate the Use of DistilBERT Using PyTorch8.1.4 Multimodal NLP: Combining Text with Images, Audio, and More8.1.5 Demonstrating the Use of OpenAI CLIP for Multimodal Understanding8.1.6 Text-to-Image Search Using OpenAI’s CLIP Model8.2 Overview of GPT8.2.1 Evolution of Text Generation Models: From RNNs to GPT8.2.2 A Short Note on the Working of GenAI (LLMs) Models8.2.3 Applications and Future Trends in Generative AI8.3 LangChain and OpenAI’s (GenAI) APIs8.3.1 Setting Up the Environment8.4 Model I/O: Easily Interface with Language Models8.4.1 Using LLMs with LangChain8.4.2 Using Chat Models with LLMsBuilding a Friendly Chat Assistant with LangChain and GPT-3.5-TurboBusiness Data Preparation for AI ApplicationsText Embedding Search with Vector Store and RetrieverSmart Query Helper with LangChain and OpenAI8.4.3 Working with Prompt Templates and Few-Shot TemplatesGuiding AI Responses with and Without TemplatesWithout TemplatesWith Prompt TemplateWith Few-Shot Template8.4.4 Parsing and SerializationInput and Output Parsing in LangChain and OpenAIInput and Output with Serialization in LangChain and OpenAI8.4.5 Customizing Model Inputs and Handling OutputsInput and Output Customization with LangChain and OpenAI8.4.6 Switching Between Different LLMsSwitching Between GPT-4 and GPT-3.5 via OpenAI API8.5 Chapter Recap
9. Advancing with LangChain and OpenAI
9.1 Data Connection with Application-Specific Data Sources9.1.1 Document Loading, Transformation, and IntegrationLangChain File Loaders for PDF, HTML, and CSVDocument Transformers: Splitting Text for NLP Processing9.1.2 Chains: Construct Sequences of Calls for Specific TasksBusiness Document Processing: Summarization, Keyword Extraction, and Sentiment Analysis9.1.3 Text Embeddings and Vector DatabasesText Similarity Using OpenAI Embeddings and Cosine Similarity9.2 AI AgentsLangChain-Based Stock Analysis AgentThe Role of LLM and Agents in This CodeRole of LLM (Large Language Model)Role of AgentsFinal Workflow9.3 Chapter Recap
10. Case Study on Symantec Analysis
10.1 Introduction10.2 Methods of Semantic Analysis10.2.1 Latent Semantic Analysis10.2.1.1 How LSA WorksTerm-Document MatrixSingular Value DecompositionStep 1. Data Collection and PreprocessingStep 2. Creating the Term-Document MatrixStep 3. Applying SVDStep 4. Dimensionality ReductionStep 5. Topic InterpretationConclusionsDimensionality ReductionCosine SimilarityLSA Applications10.3 Role of Semantic Analysis in Enhancing Customer Experience: A Case Study Fujitsu’s Kozuchi AI Agent10.3.1 About Fujitsu10.3.2 AI Initiatives of Fujitsu10.3.3 Fujitsu Kozuchi for Semantic Analysis10.3.3.1 Exhibit X.1: Kozuchi AI Agent Attending a Meeting: An Example10.3.4 Business Applications of Fujitsu Kozuchi and Its Superiority over Other Similar Tools10.4 Future Trends in Semantic AnalysisConclusion10.5 References
Index

Content preview from Mastering Text Analytics : A Hands-on Guide to NLP Using Python

S. Kadre et al.Mastering Text Analyticshttps://doi.org/10.1007/979-8-8688-1582-9_4

4. NLP Data Preprocessing Tasks with NLTK

Shailendra Kadre¹, Shailesh Kadre² and Subhendu Dey³

4.1 Why You Should Read This Chapter

Natural Language Toolkit (NLTK) is one of the most popular and useful natural language processing (NLP) platforms, useful for developing Python programs that work with natural languages (such as English, German, and others). NLTK offers user-friendly interfaces to more than 50 text corpora and other ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Text Analytics with Python: A Practitioner's Guide to Natural Language Processing

Publisher Resources

ISBN: 9798868815829Purchase Link Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Mastering Text Analytics : A Hands-on Guide to NLP Using Python

by Shailendra Kadre, Shailesh Kadre, Subhendu Dey

4. NLP Data Preprocessing Tasks with NLTK

4.1 Why You Should Read This Chapter

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.