book

Starting Data Analytics with Generative AI and Python

Name: Starting Data Analytics with Generative AI and Python
ISBN: 9781633437210

by Marian Siwiak, Artur Guja, Marlena Siwiak

November 2024

Beginner to intermediate

360 pages

12h 4m

English

Manning Publications

Read now

Unlock full access

copyright
dedication
contents
foreword
preface
acknowledgments
about this book
Who should read this bookHow to read this bookThe business data used in the bookAbout the codeliveBook discussion forum
about the authors
about the cover illustration
1 Introduction to the use of generative AI in data analytics
1.1 Inherent limitations of generative AI models1.2 The role of generative AIs in data analytics1.2.1 Generative AI in the data analytics flow1.2.2 The complementarity of language models and other data analytics tools1.2.3 Limits of generative AIs’ ability to automate and streamline data analytics processes1.3 Getting started with generative AIs for data analytics1.3.1 Web interface1.3.2 Beware of tokens1.3.3 Accessing and using the API1.3.4 Third-party integrations of generative AI models1.3.5 Running LLMs locally1.3.6 Best practices and tips for successful generative AI implementationSummary

2 Using generative AI to ensure sufficient data quality
2.1 On a whimsy of fortune2.2 A note on best practices2.3 Getting started2.4 Quality assessment structure2.4.1 Data cleaning steps2.4.2 Exploratory data analysis elements2.5 Data cleaning2.5.1 Removing duplicates2.5.2 Handling missing values2.5.3 Correcting data entry errors2.5.4 Data validation2.6 Exploratory data analysis2.6.1 Reviewing score distribution2.6.2 Time series exploration2.6.3 Mysterious variable investigation2.6.4 Harmonizing dataSummary
3 Descriptive analysis and statistical inference supported by generative AI
3.1 Research questions3.2 Analysis design3.3 Descriptive data analysis3.3.1 Popularity of product categories3.3.2 Performance of products in their categories and regions3.3.3 Review scores distribution3.3.4 Order status3.4 Inferential analysis3.4.1 Before you begin3.4.2 Relationship between product attributes and shipping costs3.4.3 Relationship between product, transaction, shipping attributes, and the review score3.4.4 Differences in sales performance and customer satisfaction between sellersSummary
4 Using generative AI for result interpretations
4.1 Problem definition4.2 Popularity of product categories4.3 Performance of products in their categories and regions4.4 Review scores distribution analysis4.5 Order status4.6 Relationship between product attributes and the shipping costs4.7 Relationship between product, transaction, shipping attributes, and the review score4.8 Differences in sales performance and customer satisfaction between sellersSummary
5 Basic text mining using generative AI
5.1 Text mining in the era of generative AI5.1.1 Generative AI is a game changer5.1.2 Beware of AI intimidation5.1.3 Unpacking the constraints5.2 Preparing for analysis5.2.1 Data quality5.2.2 Customer feedback preparation example5.3 Frequency analysis5.3.1 What can we learn from frequency analysis of customer reviews?5.3.2 Direct frequency analysis with generative AI5.3.3 Uploading a data file to ChatGPT for frequency analysis5.3.4 Extracting the most common words5.3.5 Extracting the most common phrases5.3.6 Understanding the output5.4 Co-occurrence analysis5.4.1 What can we learn from co-occurrence analysis?5.4.2 Co-occurrence analysis in practice5.4.3 Understanding the output5.5 Keyword search5.5.1 What can we learn from keyword search?5.5.2 Generating keywords with generative AI5.5.3 Generating keywords in practice5.5.4 Searching for keywords5.5.5 Improving keyword search5.5.6 Comparing generative AIs: Code snippets for positive review searches5.5.7 Seeking analytical inspiration5.6 Dictionary-based methods5.6.1 What can we learn from dictionary-based methods?5.6.2 Finding resources5.6.3 Interpreting resources5.6.4 Adapting the code to chosen resources5.6.5 Improving dictionary-based searchSummary
6 Advanced text mining with generative AI
6.1 Review analysis6.2 Sentiment analysis6.2.1 What can you learn from sentiment analysis?6.2.2 Direct sentiment analysis with generative AIs6.2.3 Sentiment analysis with generative AI’s API6.2.4 Sentiment analysis with machine learning6.2.5 Sentiment analysis with a suboptimal model6.2.6 Sentiment analysis on translated inputs6.2.7 Sentiment analysis with multilingual models6.2.8 Sentiment analysis with zero-shot learning models6.2.9 Comparing results of advanced sentiment analysis6.3 Text summarization6.3.1 How can you benefit from text summarization?6.3.2 How can generative AI help in text summarization?6.3.3 Summarizing text with ChatGPT6.3.4 Summarizing text with dedicated libraries6.3.5 Topic modelingSummary
7 Scaling and performance optimization
7.1 Performance measurement7.1.1 Execution time7.1.2 Throughput7.1.3 Resource utilization7.2 Improving code performance7.2.1 Optimizing code7.2.2 Scaling code7.3 Cloud-based deployment7.3.1 What is cloud computing?7.3.2 Moving your code to the cloud7.4 Code conversionSummary
8 Risk, mitigation, and tradeoffs
8.1 The risks of GenAI, in context8.2 General best practices8.2.1 AI use policy8.2.2 Encouraging transparency and accountability8.2.3 Educating stakeholders8.2.4 Validating model outputs with expert knowledge8.3 AI delusion and hallucination risks8.4 Mitigating misinterpretation and miscommunication risks8.4.1 Ensuring contextual understanding8.4.2 Tailoring model prompts and iterative query refinement8.4.3 Implementing post-processing techniques8.4.4 Implementing best practices for clearly communicating results8.4.5 Establishing a feedback loop8.5 Model bias and fairness risks8.5.1 Recognizing and identifying bias in model outputs8.5.2 Applying bias detection and mitigation techniques8.5.3 Encouraging diversity and ethical use of generative AIs8.5.4 Continuously monitoring and updating models8.6 Privacy and security risks8.6.1 Identifying sensitive data8.6.2 Data anonymization and pseudonymization8.6.3 Social engineering and phishing8.6.4 Compliance with data protection regulations8.6.5 Regular security audits and assessments8.6.6 Employee training and awareness8.7 Legal and compliance risks8.7.1 Understanding applicable regulations8.7.2 Intellectual property and licensing8.7.3 Transparency and explainability8.7.4 Establishing a compliance framework8.7.5 Regularly reviewing and updating compliance practices8.8 Emergent risks8.8.1 Rogue models8.8.2 Vulnerable crown jewels8.8.3 Unknown unknownsSummary
appendix A Specifying multiple DataFrames to ChatGPT v4
A.1 Conversation recorded on April 1, 2023
appendix B On debugging ChatGPT’s code
B.1 Conversation recorded on April 3, 2023
appendix C On laziness and human errors
C.1 Conversation recorded on April 7, 2023
index

Content preview from Starting Data Analytics with Generative AI and Python

7 Scaling and performance optimization

This chapter covers

Optimizing data analysis code
Scaling code across multiple CPUs and GPUs
Code conversion between programming languages
Cloud-based deployment
Using the Dask framework for all of this

In today’s data-driven world, it’s more likely than not that the insights required by your business will be gained from vast amounts of data. As data volumes continue to grow and analytical complexity increases, and as algorithms and data processing pipelines reach the point where the sheer volume of data or the complexity of operations limits their ability to deliver timely and accurate results, you’ll encounter the need for code performance optimization. It will likely become essential for ensuring ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Analytical Skills for AI and Data Science

Publisher Resources

ISBN: 9781633437210Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Starting Data Analytics with Generative AI and Python

by Marian Siwiak, Artur Guja, Marlena Siwiak