book

Mastering NLP from Foundations to LLMs

Name: Mastering NLP from Foundations to LLMs
ISBN: 9781804619186

by Lior Gazit, Meysam Ghaffari

April 2024

Beginner to intermediate

340 pages

10h 45m

English

Packt Publishing

Read now

Unlock full access

Mastering NLP from Foundations to LLMs
ForewordContributorsAbout the authorsAbout the reviewersDisclaimer
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchReviewsShare Your ThoughtsDownload a free PDF copy of this book
Chapter 1: Navigating the NLP Landscape: A Comprehensive Introduction
Who this book is forWhat is natural language processing?The history and evolution of natural language processingInitial strategies in the machine processing of natural languageA winning synergy – the coming together of NLP and MLIntroduction to math and statistics in NLPUnderstanding language models – ChatGPT exampleSummaryQuestions and answers
Chapter 2: Mastering Linear Algebra, Probability, and Statistics for Machine Learning and NLP
Introduction to linear algebraBasic operations on matrices and vectorsMatrix definitionsEigenvalues and eigenvectorsNumerical methods for finding eigenvectorsEigenvalue decompositionSingular value decompositionBasic probability for machine learningStatistically independentDiscrete random variables and their distributionProbability density functionBayesian estimationSummaryFurther readingReferences
Chapter 3: Unleashing Machine Learning Potentials in Natural Language Processing
Technical requirementsData explorationData visualizationData cleaningFeature selectionFeature engineeringCommon machine learning modelsLinear regressionLogistic regressionDecision treesRandom forestSupport vector machines (SVMs)Neural networks and transformersModel underfitting and overfittingSplitting dataHyperparameter tuningEnsemble modelsBaggingBoostingStackingRandom forestsGradient boostingHandling imbalanced dataSMOTEThe NearMiss algorithmCost-sensitive learningData augmentationDealing with correlated dataSummaryReferences
Chapter 4: Streamlining Text Preprocessing Techniques for Optimal NLP Performance
Technical requirementsLowercasing in NLPRemoving special characters and punctuationStop word removalNERPOS taggingRule-based methodsStatistical methodsDeep learning-based methodsRegular expressionsTokenizationExplaining the preprocessing pipelineCode for NER and POSSummary
Chapter 5: Empowering Text Classification: Leveraging Traditional Machine Learning Techniques
Technical requirementsTypes of text classificationSupervised learningUnsupervised learningSemi-supervised learningSentence classification using one-hot encoding vector representationText classification using TF-IDFText classification using Word2VecWord2VecModel evaluationOverfitting and underfittingHyperparameter tuningAdditional topics in applied text classificationTopic modeling – a particular use case of unsupervised text classificationLDAReal-world ML system design for NLP text classificationImplementing an ML solutionReviewing our use case – ML system design for NLP classification in a Jupyter NotebookThe pipelineCode settingsFeature selectionGenerating the chosen modelSummary
Chapter 6: Text Classification Reimagined: Delving Deep into Deep Learning Language Models
Technical requirementsUnderstanding deep learning basicsWhat is a neural network?The basic design of a neural networkNeural network common termsThe architecture of different neural networksThe challenges of training neural networksLanguage modelsSemi-supervised learningUnsupervised learningTransfer learningUnderstanding transformersArchitecture of transformersApplications of transformersLearning more about large language modelsThe challenges of training language modelsSpecific designs of language modelsChallenges of using GPT-3Reviewing our use case – ML/DL system design for NLP classification in a Jupyter NotebookThe business objectiveThe technical objectiveThe pipelineSummary
Chapter 7: Demystifying Large Language Models: Theory, Design, and Langchain Implementation
Technical requirementsWhat are LLMs and how are they different from LMs?n-gram modelsHidden Markov models (HMMs)Recurrent neural networks (RNNs)How LLMs stand outMotivations for developing and using LLMsImproved performanceBroad generalizationFew-shot learningUnderstanding complex contextsMultilingual capabilitiesHuman-like text generationChallenges in developing LLMsAmounts of dataComputational resourcesRisk of biasModel robustnessInterpretability and debuggingEnvironmental impactDifferent types of LLMsTransformer modelsExample designs of state-of-the-art LLMsGPT-3.5 and ChatGPTLM pretrainingTraining the reward modelHow to fine-tune the model using reinforcement learningGPT-4LLaMAPaLMOpen-source tools for RLHFSummaryReferences
Chapter 8: Accessing the Power of Large Language Models: Advanced Setup and Integration with RAG
Technical requirementsSetting up an LLM application – API-based closed source modelsChoosing a remote LLM providerPrompt engineering and priming GPTExperimenting with OpenAI’s GPT modelSetting up an LLM application – local open source modelsAbout the different aspects that distinguish between open source and closed sourceHugging Face’s hub of modelsEmploying LLMs from Hugging Face via PythonExploring advanced system design – RAG and LangChainLangChain’s design conceptsData sourcesData that is not pre-embeddedChainsAgentsLong-term memory and referring to prior conversationsEnsuring continuous relevance through incremental updates and automated monitoringReviewing a simple LangChain setup in a Jupyter notebookSetting up a LangChain pipeline with PythonLLMs in the cloudAWSMicrosoft AzureGCPConcluding cloud servicesSummary

Chapter 9: Exploring the Frontiers: Advanced Applications and Innovations Driven by LLMs
Technical requirementsEnhancing LLM performance with RAG and LangChain – a dive into advanced functionalitiesLangChain pipeline with Python – enhancing performance with LLMsAdvanced methods with chainsAsking the LLM a general knowledge questionRequesting output structure – making the LLM provide output in a particular data formatEvolving to a fluent conversation – inserting an element of memory to have previous interactions as reference and context for follow-up promptsRetrieving information from various web sources automaticallyRetrieving content from a YouTube video and summarizing itPrompt compression and API cost reductionPrompt compressionExperimenting with prompt compression and evaluating trade-offsMultiple agents – forming a team of LLMs that collaboratePotential advantages of multiple LLM agents working simultaneouslyConcluding thoughts on the multiple-agent teamSummary
Chapter 10: Riding the Wave: Analyzing Past, Present, and Future Trends Shaped by LLMs and AI
Key technical trends around LLMs and AIComputation power – the engine behind LLMsThe future of computational power in NLPLarge datasets and their indelible mark on NLP and LLMsPurpose – training, benchmarking, and domain expertiseValue – robustness, diversity, and efficiencyImpact – democratization, proficiency, and new concernsEvolution of large language models – purpose, value, and impactPurpose – why the push for bigger and better LLMs?Value – the LLM advantageImpact – changing the landscapeNLP and LLMs in the business worldBusiness sectorsCustomer interactions and service – the early adopterChange management driven by AI’s impactBehavioral trends induced by AI and LLMs – the social aspectPersonal assistants becoming indispensableEase in communication and bridging language barriersEthical implications of delegated decisionsEthics and risks – growing concerns around the implementation of AISummary
Chapter 11: Exclusive Industry Insights: Perspectives and Predictions from World Class Experts
Overview of our expertsNitzan Mekel-Bobrov, PhDDavid Sontag, PhDJohn D. Halamka, M.D., M.S.Xavier Amatriain, PhDMelanie Garson, PhDOur questions and the experts’ answersNitzan Mekel-BobrovQ1.1 – Future of LLM – hybrid learning paradigms: In light of the evolving landscape of learning schemes, what do you envision as the next breakthrough in combining different learning paradigms within LLMs?Q2.1 – As the Chief AI Officer becomes more integral to the corporate hierarchy, what unique challenges do you foresee in bridging the gap between AI potential and practical business applications, and how should the CAIO’s role evolve to meet these challenges?Q3 – How do foundation models and the strategies of major tech companies toward open sourcing affect data ownership and its value for businesses?David SontagQ1 – As we progress toward creating more equitable and unbiased datasets, what strategies do you believe are most effective in identifying and mitigating implicit biases within large datasets?Q2 – How do you see these strategies evolving with the advancement of NLP technologies, and what do you envision as the next breakthrough in combining different learning paradigms within LLMs?Q3 – We are witnessing an ongoing evolution of regulations around AI from the aspects of training data and model usage. What are the implications for the future development of LLMs in this regulated landscape?John D. HalamkaQ2 – AI-driven organizational structure – in what ways do you predict AI will continue to reshape companies’ organizational structures to maximize the benefits of AI?Q3 – Ethical concerns and strategies to combat over-delegation – as AI continues to penetrate daily decision-making processes, what strategies do you recommend to prevent overreliance on AI systems and to maintain a healthy level of human critical thinking and autonomy?Xavier AmatriainQ1.1 – The future of LLM – hybrid learning paradigms: In light of the evolving landscape of learning schemes, what do you envision as the next breakthrough in combining different learning paradigms within LLMs?Q2 – AI-driven organizational structure – in what ways do you predict AI will continue to reshape internal business operations, and how should companies prepare to adapt their organizational structures to maximize the benefits of AI, especially in decision-making and operational efficiency?Melanie GarsonQ1 – As this book is designed to address technical practitioners in the world of ML and AI, what value would they find in being aware of the various legal and regulatory aspects?Q2 – From the perspective of a legal expert, how can we categorize the diverse array of risks associated with the burgeoning advancements in AI technology?Q3 – How can the development and deployment of AI and LLMs be guided to mitigate ethical concerns such as bias and ensure their responsible use in decision-making processes, particularly in high-risk and regulated industries?Q4 – What strategies can be implemented to transition from traditional roles to collaborative human-AI teams, ensuring the development of human expertise alongside AI integration in the workplace?Summary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Overview

In "Mastering NLP from Foundations to LLMs," you'll embark on a comprehensive learning journey through Natural Language Processing (NLP). This book masterfully guides readers from the basics of machine learning and text data preprocessing to advanced topics like large language models (LLMs), all supported by practical Python examples. Enhance your ability to solve real-world business problems using NLP.

What this Book will help me do

Build a strong foundation in the mathematical principles behind machine learning and NLP.
Develop skillsets necessary to preprocess and analyze text data for NLP projects.
Create and deploy machine learning-based NLP systems using Python efficiently.
Understand the inner workings of large language models (LLMs) and implement them in practical applications.
Gain insights into the future trends and emerging innovations in the NLP and AI domains.

Author(s)

None Gazit and Meysam Ghaffari are recognized experts in the field of NLP and machine learning. With years of collective experience in academia and industry, they bring a detail-oriented and practical approach to teaching complex technical subjects. Their passion for sharing knowledge ensures each reader achieves mastery over the challenging topics.

Who is it for?

This book is ideal for machine learning and deep learning researchers, students in STEM fields, and NLP practitioners looking to enhance their expertise in NLP techniques and tools. Individuals with a basic understanding of Python and machine learning concepts will find this book an invaluable resource to expand their knowledge of NLP workflows. Furthermore, professionals seeking to implement NLP-driven solutions in their projects will benefit greatly from the practical insights and detailed guidance provided.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Introduction to Transformer Models for NLP: Using BERT, GPT, and More to Solve Modern Natural Language Processing Tasks

Publisher Resources

ISBN: 9781804619186

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills