book

Python Data Analysis - Third Edition

Name: Python Data Analysis - Third Edition
ISBN: 9781789955248

by Avinash Navlani, Ivan Idris

February 2021

Beginner to intermediate

478 pages

8h 38m

English

Packt Publishing

Read now

Unlock full access

Title Page
Copyright and Credits
Python Data Analysis Third Edition
About Packt
Why subscribe?
Contributors
About the authorsAbout the reviewersPackt is searching for authors like you
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Section 1: Foundation for Data Analysis
Getting Started with Python Libraries
Understanding data analysisThe standard process of data analysisThe KDD processSEMMA CRISP-DMComparing data analysis and data scienceThe roles of data analysts and data scientistsThe skillsets of data analysts and data scientistsInstalling Python 3Python installation and setup on WindowsPython installation and setup on LinuxPython installation and setup on Mac OS X with a GUI installerPython installation and setup on Mac OS X with brewSoftware used in this bookUsing IPython as a shellReading manual pagesWhere to find help and references to Python data analysis librariesUsing JupyterLabUsing Jupyter NotebooksAdvanced features of Jupyter NotebooksKeyboard shortcutsInstalling other kernelsRunning shell commandsExtensions for NotebookSummary
NumPy and pandas
Technical requirementsUnderstanding NumPy arraysArray featuresSelecting array elementsNumPy array numerical data typesdtype objectsData type character codesdtype constructorsdtype attributesManipulating array shapesThe stacking of NumPy arraysPartitioning NumPy arraysChanging the data type of NumPy arraysCreating NumPy views and copiesSlicing NumPy arraysBoolean and fancy indexingBroadcasting arraysCreating pandas DataFramesUnderstanding pandas SeriesReading and querying the Quandl dataDescribing pandas DataFramesGrouping and joining pandas DataFrameWorking with missing valuesCreating pivot tablesDealing with datesSummaryReferences
Statistics
Technical requirementsUnderstanding attributes and their typesTypes of attributesDiscrete and continuous attributesMeasuring central tendencyMeanModeMedianMeasuring dispersionSkewness and kurtosisUnderstanding relationships using covariance and correlation coefficientsPearson's correlation coefficientSpearman's rank correlation coefficientKendall's rank correlation coefficientCentral limit theoremCollecting samplesPerforming parametric testsPerforming non-parametric tests Summary
Linear Algebra
Technical requirementsFitting to polynomials with NumPyDeterminantFinding the rank of a matrixMatrix inverse using NumPySolving linear equations using NumPyDecomposing a matrix using SVDEigenvectors and Eigenvalues using NumPyGenerating random numbersBinomial distributionNormal distributionTesting normality of data using SciPyCreating a masked array using the numpy.ma subpackageSummary

Section 2: Exploratory Data Analysis and Data Cleaning
Data Visualization
Technical requirementsVisualization using MatplotlibAccessories for chartsScatter plotLine plotPie plotBar plotHistogram plotBubble plotpandas plottingAdvanced visualization using the Seaborn packagelm plotsBar plotsDistribution plotsBox plotsKDE plotsViolin plotsCount plotsJoint plotsHeatmapsPair plotsInteractive visualization with BokehPlotting a simple graphGlyphsLayoutsNested layout using row and column layoutsMultiple plotsInteractionsHide click policyMute click policyAnnotationsHover toolWidgetsTab panelSliderSummary
Retrieving, Processing, and Storing Data
Technical requirementsReading and writing CSV files with NumPyReading and writing CSV files with pandasReading and writing data from ExcelReading and writing data from JSONReading and writing data from HDF5Reading and writing data from HTML tablesReading and writing data from ParquetReading and writing data from a pickle pandas objectLightweight access with sqllite3Reading and writing data from MySQLInserting a whole DataFrame into the databaseReading and writing data from MongoDBReading and writing data from CassandraReading and writing data from RedisPonyORMSummary
Cleaning Messy Data
Technical requirementsExploring dataFiltering data to weed out the noiseColumn-wise filtration Row-wise filtration Handling missing valuesDropping missing valuesFilling in a missing valueHandling outliersFeature encoding techniquesOne-hot encodingLabel encodingOrdinal encoderFeature scalingMethods for feature scalingFeature transformationFeature splittingSummary
Signal Processing and Time Series
Technical requirementsThe statsmodels modulesMoving averagesWindow functionsDefining cointegrationSTL decompositionAutocorrelationAutoregressive modelsARMA modelsGenerating periodic signalsFourier analysisSpectral analysis filteringSummary
Section 3: Deep Dive into Machine Learning
Supervised Learning - Regression Analysis
Technical requirementsLinear regressionMultiple linear regressionUnderstanding multicollinearityRemoving multicollinearityDummy variablesDeveloping a linear regression modelEvaluating regression model performanceR-squaredMSEMAERMSEFitting polynomial regressionRegression models for classificationLogistic regressionCharacteristics of the logistic regression modelTypes of logistic regression algorithmsAdvantages and disadvantages of logistic regressionImplementing logistic regression using scikit-learnSummary
Supervised Learning - Classification Techniques
Technical requirementsClassificationNaive Bayes classificationDecision tree classificationKNN classificationSVM classificationTerminologySplitting training and testing setsHoldoutK-fold cross-validationBootstrap methodEvaluating the classification model performanceConfusion matrixAccuracyPrecisionRecallF-measureROC curve and AUCSummary
Unsupervised Learning - PCA and Clustering
Technical requirementsUnsupervised learningReducing the dimensionality of dataPCAPerforming PCAClusteringFinding the number of clustersThe elbow methodThe silhouette methodPartitioning data using k-means clusteringHierarchical clusteringDBSCAN clusteringSpectral clusteringEvaluating clustering performanceInternal performance evaluationThe Davies-Bouldin indexThe silhouette coefficientExternal performance evaluationThe Rand scoreThe Jaccard scoreF-Measure or F1-scoreThe Fowlkes-Mallows scoreSummary
Section 4: NLP, Image Analytics, and Parallel Computing
Analyzing Textual Data
Technical requirementsInstalling NLTK and SpaCyText normalizationTokenizationRemoving stopwordsStemming and lemmatizationPOS taggingRecognizing entitiesDependency parsingCreating a word cloudBag of WordsTF-IDFSentiment analysis using text classificationClassification using BoWClassification using TF-IDFText similarityJaccard similarityCosine similaritySummary
Analyzing Image Data
Technical requirementsInstalling OpenCVUnderstanding image dataBinary imagesGrayscale imagesColor imagesColor modelsDrawing on imagesWriting on imagesResizing imagesFlipping imagesChanging the brightnessBlurring an imageFace detectionSummary
Parallel Computing Using Dask
Parallel computing using DaskDask data typesDask ArraysDask DataFramesDataFrame IndexingFilter dataGroupbyConverting a pandas DataFrame into a Dask DataFrameConverting a Dask DataFrame into a pandas DataFrameDask BagsCreating a Dask Bag using Python iterable itemsCreating a Dask Bag using a text fileStoring a Dask Bag in a text fileStoring a Dask Bag in a DataFrameDask DelayedPreprocessing data at scaleFeature scaling in DaskFeature encoding in DaskMachine learning at scaleParallel computing using scikit-learnReimplementing ML algorithms for DaskLogistic regressionClusteringSummary
Other Books You May Enjoy
Leave a review - let other readers know what you think

Content preview from Python Data Analysis - Third Edition

Retrieving, Processing, and Storing Data

Data can be found everywhere, in all shapes and forms. We can get it from the web, IoT sensors, emails, FTP, and databases. We can also collect it ourselves in a lab experiment, election polls, marketing polls, and social surveys. As a data professional, you should know how to handle a variety of datasets as that is a very important skill. We will discuss retrieving, processing, and storing various types of data in this chapter. This chapter offers an overview of how to acquire data in various formats, such as CSV, Excel, JSON, HDF5, Parquet, and pickle.

Sometimes, we need to store or save the data before or after the data analysis. We will also learn how to access data from relational and NoSQL

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781789955248

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Data Analysis - Third Edition

by Avinash Navlani, Ivan Idris

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.