book

Python: Real-World Data Science

by Dusty Phillips, Fabrizio Romano, Phuong Vo.T.H, Martin Czygan, Robert Layton, Sebastian Raschka

June 2016

Beginner to intermediate

1255 pages

29h 1m

English

Packt Publishing

Read now

Unlock full access

A proper introduction

PortabilityCoherenceDeveloper productivityAn extensive librarySoftware qualitySoftware integrationSatisfaction and enjoyment
Python 2 versus Python 3 – the great debate
Installing PythonInstalling IPythonInstalling additional packages
Running Python scriptsRunning the Python interactive shellRunning Python as a serviceRunning Python as a GUI application
How do we use modules and packages
Names and namespacesScopes
Introducing object-oriented
Data describes objectsBehaviors are actions
Inheritance provides abstractionMultiple inheritance
Creating Python classesAdding attributesMaking it do somethingTalking to yourselfMore argumentsInitializing the objectExplaining yourself
Organizing the modulesAbsolute importsRelative imports
Basic inheritanceExtending built-insOverriding and super
The diamond problemDifferent sets of arguments
Using an abstract base classCreating an abstract base classDemystifying the magic
Raising exceptionsRaising an exceptionThe effects of an exceptionHandling exceptionsThe exception hierarchyDefining our own exceptions
Treat objects as objects
Properties in detailDecorators – another way to create propertiesDeciding when to use properties
Removing duplicate codeIn practice
Empty objects
Named tuples
Dictionary use casesUsing defaultdictCounter
Sorting lists
FIFO queuesLIFO queuesPriority queues
Python built-in functionsThe len() functionReversedEnumerateFile I/OPlacing it in context
Default argumentsVariable argument listsUnpacking arguments
Using functions as attributesCallable objects
StringsString manipulationString formattingEscaping bracesKeyword argumentsContainer lookupsObject lookupsMaking it look rightStrings are UnicodeConverting bytes to textConverting text to bytesMutable byte strings
Matching patternsMatching a selection of charactersEscaping charactersMatching multiple charactersGrouping patterns togetherGetting information from regular expressionsMaking repeated regular expressions efficient
Customizing picklesSerializing web objects
Design patterns in brief
The iterator protocol
List comprehensionsSet and dictionary comprehensionsGenerator expressions
Yield items from another iterable
Back to log parsingClosing coroutines and throwing exceptionsThe relationship between coroutines, generators, and functions
The decorator patternA decorator exampleDecorators in Python
An observer example
A strategy exampleStrategy in Python
A state exampleState versus strategyState transition as coroutines
Singleton implementation
A template example
The adapter pattern
Why test?Test-driven development
Assertion methodsReducing boilerplate and cleaning upOrganizing and running testsIgnoring broken tests
One way to do setup and cleanupA completely different way to set up variablesSkipping tests with py.test
Implementing it
ThreadsThe many problems with threadsShared memoryThe global interpreter lockThread overhead
Multiprocessing poolsQueuesThe problems with multiprocessing
AsyncIO in actionReading an AsyncIO futureAsyncIO for networkingUsing executors to wrap blocking codeStreamsExecutors
Data analysis and processing
NumPypandasMatplotlibPyMongoThe scikit-learn library
NumPy arraysData typesArray creationIndexing and slicingFancy indexingNumerical operations on arrays
Loading and saving dataSaving an arrayLoading an array
An overview of the pandas package
SeriesThe DataFrame
Reindexing and altering labelsHead and tailBinary operationsFunctional statisticsFunction applicationSorting
Hierarchical indexingThe Panel data
The matplotlib API primerLine propertiesFigures and subplots
Scatter plotsBar plotsContour plotsHistogram plots
BokehMayaVi
Time series primer
Interacting with data in text formatReading data from text formatWriting data to text format
HDF5
The simple valueListSetOrdered set
Data mungingCleaning dataFilteringMerging dataReshaping data
Introducing data mining
What is affinity analysis?Product recommendationsLoading the dataset with NumPyImplementing a simple ranking of rulesRanking to find the best rules
Loading and preparing the datasetImplementing the OneR algorithmTesting the algorithm
scikit-learn estimatorsNearest neighborsDistance metricsLoading the datasetMoving towards a standard workflowRunning the algorithmSetting parameters
An exampleStandard preprocessingPutting it all together
Loading the datasetCollecting the dataUsing pandas to load the datasetCleaning up the datasetExtracting new features
Parameters in decision treesUsing decision trees
Putting it all together
How do ensembles work?Parameters in Random forestsApplying Random forestsEngineering new features
Affinity analysisAlgorithms for affinity analysisChoosing parameters
Obtaining the datasetLoading with pandasSparse data formats
The Apriori algorithmImplementation
Evaluation
Feature extractionRepresenting reality in modelsCommon feature patternsCreating good features
Selecting the best individual features
The transformer APIImplementation detailsUnit testingPutting it all together
DisambiguationDownloading data from a social networkLoading and classifying the datasetCreating a replicable dataset from Twitter
Bag-of-wordsN-gramsOther features
Bayes' theoremNaive Bayes algorithmHow it works
Extracting word countsConverting dictionaries to a matrixTraining the Naive Bayes classifierPutting it all togetherEvaluation using the F1-scoreGetting useful features from models
Loading the datasetClassifying with an existing modelGetting follower information from TwitterBuilding the networkCreating a graphCreating a similarity graph
Connected componentsOptimizing criteria
Artificial neural networksAn introduction to neural networks
Drawing basic CAPTCHAsSplitting the image into individual lettersCreating a training datasetAdjusting our training dataset to our methodology
Back propagationPredicting words
Ranking mechanisms for wordsPutting it all together
Attributing documents to authorsApplications and use casesAttributing authorshipGetting the data
Counting function wordsClassifying with function words
Classifying with SVMsKernels
Extracting character n-grams
Accessing the Enron datasetCreating a dataset loaderPutting it all togetherEvaluation
Obtaining news articlesUsing a Web API to get dataReddit as a data sourceGetting the data
Finding the stories in arbitrary websitesPutting it all together
The k-means algorithmEvaluating the resultsExtracting topic information from clustersUsing clustering algorithms as transformers
Evidence accumulationHow it worksImplementation
An introduction to online learningImplementation
Object classification
Use cases
IntuitionImplementationAn introduction to TheanoAn introduction to LasagneImplementing neural networks with nolearn
When to use GPUs for computationRunning our code on a GPU
Getting the dataCreating the neural networkPutting it all together
Big data
IntuitionA word count exampleHadoop MapReduce
Getting the dataNaive Bayes predictionThe mrjob packageExtracting the blog postsTraining Naive BayesPutting it all togetherTraining on Amazon's EMR infrastructure
Chapter 1 – Getting Started with Data MiningScikit-learn tutorialsExtending the IPython Notebook
More complex pipelinesComparing classifiers
More on pandas
The Eclat algorithm
Vowpal Wabbit
Natural language processing and part-of-speech tagging
More complex algorithms
Deeper networksReinforcement learning
Local n-grams
Real-time clusterings
Keras and Pylearn2Mahotas
Courses on HadoopPydoopRecommendation engine
How to transform data into knowledge
Making predictions about the future with supervised learningClassification for predicting class labelsRegression for predicting continuous outcomesSolving interactive problems with reinforcement learningDiscovering hidden structures with unsupervised learningFinding subgroups with clusteringDimensionality reduction for data compression
Preprocessing – getting data into shapeTraining and selecting a predictive modelEvaluating models and predicting unseen data instances
Artificial neurons – a brief glimpse into the early history of machine learning
Training a perceptron model on the Iris dataset
Minimizing cost functions with gradient descentImplementing an Adaptive Linear Neuron in PythonLarge scale machine learning and stochastic gradient descent
Choosing a classification algorithm
Training a perceptron via scikit-learn
Logistic regression intuition and conditional probabilitiesLearning the weights of the logistic cost functionTraining a logistic regression model with scikit-learnTackling overfitting via regularization
Maximum margin intuitionDealing with the nonlinearly separable case using slack variablesAlternative implementations in scikit-learn
Using the kernel trick to find separating hyperplanes in higher dimensional space
Maximizing information gain – getting the most bang for the buckBuilding a decision treeCombining weak to strong learners via random forests
Dealing with missing dataEliminating samples or features with missing valuesImputing missing valuesUnderstanding the scikit-learn estimator API
Mapping ordinal featuresEncoding class labelsPerforming one-hot encoding on nominal features
Sparse solutions with L1 regularizationSequential feature selection algorithms
Unsupervised dimensionality reduction via principal component analysisTotal and explained varianceFeature transformationPrincipal component analysis in scikit-learn
Computing the scatter matricesSelecting linear discriminants for the new feature subspaceProjecting samples onto the new feature spaceLDA via scikit-learn
Kernel functions and the kernel trickImplementing a kernel principal component analysis in PythonExample 1 – separating half-moon shapesExample 2 – separating concentric circlesProjecting new data pointsKernel principal component analysis in scikit-learn
Streamlining workflows with pipelinesLoading the Breast Cancer Wisconsin datasetCombining transformers and estimators in a pipeline
The holdout methodK-fold cross-validation
Diagnosing bias and variance problems with learning curvesAddressing overfitting and underfitting with validation curves
Tuning hyperparameters via grid searchAlgorithm selection with nested cross-validation
Reading a confusion matrixOptimizing the precision and recall of a classification modelPlotting a receiver operating characteristicThe scoring metrics for multiclass classification
Learning with ensembles
Combining different algorithms for classification with majority vote
Introducing a simple linear regression model
Visualizing the important characteristics of a dataset
Solving regression for regression parameters with gradient descentEstimating the coefficient of a regression model via scikit-learn
Modeling nonlinear relationships in the Housing DatasetDealing with nonlinear relationships using random forestsDecision tree regressionRandom forest regression
Module 2: Data AnalysisChapter 1: Introducing Data Analysis and LibrariesChapter 2: Object-oriented DesignChapter 3: Data Analysis with pandasChapter 4: Data VisualizationChapter 5: Time SeriesChapter 6: Interacting with DatabasesChapter 7: Data Analysis Application Examples
Chapter 1: Getting Started with Data MiningChapter 2: Classifying with scikit-learn EstimatorsChapter 3: Predicting Sports Winners with Decision TreesChapter 4: Recommending Movies Using Affinity AnalysisChapter 5: Extracting Features with TransformersChapter 6: Social Media Insight Using Naive BayesChapter 7: Discovering Accounts to Follow Using Graph MiningChapter 8: Beating CAPTCHAs with Neural NetworksChapter 9: Authorship AttributionChapter 10: Clustering News ArticlesChapter 11: Classifying Objects in Images Using Deep LearningChapter 12: Working with Big Data
Chapter 1: Giving Computers the Ability to Learn from DataChapter 2: Training Machine LearningChapter 3: A Tour of Machine Learning Classifiers Using scikit-learnChapter 4: Building Good Training Sets – Data PreprocessingChapter 5: Compressing Data via Dimensionality ReductionChapter 6: Learning Best Practices for Model Evaluation and Hyperparameter TuningChapter 7: Combining Different Models for Ensemble LearningChapter 8: Predicting Continuous Target Variables with Regression Analysis

Content preview from Python: Real-World Data Science

Implementing an ordinary least squares linear regression model

At the beginning of this chapter, we discussed that linear regression can be understood as finding the best-fitting straight line through the sample points of our training data. However, we have neither defined the term best-fitting nor have we discussed the different techniques of fitting such a model. In the following subsections, we will fill in the missing pieces of this puzzle using the Ordinary Least Squares (OLS) method to estimate the parameters of the regression line that minimizes the sum of the squared vertical distances (residuals or errors) to the sample points.