book

Python for Data Science

by Yuli Vasiliev

August 2022

Beginner to intermediate

248 pages

6h 3m

English

No Starch Press

Read now

Unlock full access

Using Python for Data ScienceWho Should Read This Book? What’s in the Book?
Categories of DataUnstructured DataStructured DataSemistructured DataTime Series DataSources of DataAPIsWeb PagesDatabasesFilesThe Data Processing PipelineAcquisitionCleansingTransformationAnalysisStorageThe Pythonic WaySummary
ListsCreating a ListUsing Common List Object MethodsUsing Slice NotationUsing a List as a QueueUsing a List as a StackUsing Lists and Stacks for Natural Language ProcessingMaking Improvements with List ComprehensionsTuplesA List of TuplesImmutabilityDictionariesA List of DictionariesAdding to a Dictionary with setdefault()Loading JSON into a DictionarySetsRemoving Duplicates from SequencesPerforming Common Set OperationsExercise #1: Improved Photo Tag AnalysisSummary
NumPyInstalling NumPyCreating a NumPy ArrayPerforming Element-Wise OperationsUsing NumPy Statistical FunctionsExercise #2: Using NumPy Statistical Functionspandaspandas Installationpandas SeriesExercise #3: Combining Three Seriespandas DataFramesExercise #4: Using Different Joinsscikit-learnInstalling scikit-learnObtaining a Sample DatasetLoading the Sample Dataset into a pandas DataFrameSplitting the Sample Dataset into a Training Set and a Test SetTransforming Text into Numerical Feature VectorsTraining and Evaluating the ModelMaking Predictions on New DataSummary
Importing Data Using Python’s open() FunctionText FilesTabular Data FilesExercise #5: Opening JSON FilesBinary FilesExporting Data to FilesAccessing Remote Files and APIsHow HTTP Requests WorkThe urllib3 LibraryThe Requests LibraryExercise #6: Accessing an API with RequestsMoving Data to and from a DataFrameImporting Nested JSON StructuresConverting a DataFrame to JSONExercise #7: Manipulating Complex JSON StructuresLoading Online Data into a DataFrame with pandas-datareaderSummary
Relational DatabasesUnderstanding SQL StatementsGetting Started with MySQLDefining the Database StructureInserting Data into the DatabaseQuerying Database DataExercise #8: Performing a One-to-Many JoinUsing Database Analytics ToolsNoSQL DatabasesKey-Value StoresDocument-Oriented DatabasesExercise #9: Inserting and Querying Multiple DocumentsSummary
Data to AggregateCombining DataFramesGrouping and Aggregating the DataViewing Specific Aggregations by MultiIndexSlicing a Range of Aggregated ValuesSlicing Within Aggregation LevelsAdding a Grand TotalAdding SubtotalsExercise #10: Excluding Total Rows from the DataFrameSelecting All Rows in a GroupSummary

Combining Built-in Data StructuresCombining Lists and Tuples with +Combining Dictionaries with **Combining Corresponding Rows from Two StructuresImplementing Different Types of Joins for ListsConcatenating NumPy ArraysExercise #11: Adding New Rows/Columns to a NumPy ArrayCombining pandas Data StructuresConcatenating DataFramesJoining Two DataFramesSummary
Common VisualizationsLine GraphsBar GraphsPie ChartsHistogramsPlotting with MatplotlibInstalling MatplotlibUsing matplotlib.pyplotWorking with Figure and Axes ObjectsExercise #12: Combining Bins into an “Other” SliceUsing Other Libraries with MatplotlibPlotting pandas DataPlotting Geospatial Data with CartopyExercise #13: Drawing a Map with Cartopy and MatplotlibSummary
Obtaining Location DataTurning a Human-Readable Address into Geo CoordinatesGetting the Geo Coordinates of a Moving ObjectSpatial Data Analysis with geopy and ShapelyFinding the Closest ObjectFinding Objects in a Certain AreaExercise #14: Defining Two or More PolygonsCombining Both ApproachesExercise #15: Further Improving the Pick-Up AlgorithmCombining Spatial and Nonspatial DataDeriving Nonspatial AttributesExercise #16: Filtering Data with a List ComprehensionJoining Spatial and Nonspatial DatasetsSummary
Regular vs. Irregular Time SeriesCommon Time Series Analysis TechniquesCalculating Percentage ChangesRolling Window CalculationsCalculating the Percentage Change of a Rolling AverageMultivariate Time SeriesProcessing Multivariate Time SeriesAnalyzing Dependencies Between VariablesExercise #17: Adding More Metrics to Analyze DependenciesSummary
Association RulesSupportConfidenceLiftThe Apriori AlgorithmCreating a Transaction DatasetIdentifying Frequent ItemsetsGenerating Association RulesVisualizing Association RulesGaining Actionable Insights from Association RulesGenerating RecommendationsPlanning Discounts Based on Association RulesExercise #18: Mining Real Transaction DataSummary
Why Machine Learning?Types of Machine LearningSupervised LearningUnsupervised LearningHow Machine Learning WorksData to Learn FromA Statistical ModelPreviously Unseen DataA Sentiment Analysis Example: Classifying Product ReviewsObtaining Product ReviewsCleansing the DataSplitting and Transforming the DataTraining the ModelEvaluating the ModelExercise #19: Expanding the Example SetPredicting Stock TrendsGetting DataDeriving Features from Continuous DataGenerating the Output VariableTraining and Evaluating the ModelExercise #20: Experimenting with Different Stocks and New MetricsSummary

Content preview from Python for Data Science

4 Accessing Data from Files and APIs

Accessing data and getting it into your script is the first step of data analysis. This chapter covers several ways of importing data from files and other sources into your Python application, as well as ways to export data to files. You’ll see how to access the content of different types of files, including those stored locally on your machine and others accessible remotely through HTTP requests. You’ll also learn how to get data by sending requests to APIs that are accessible via a URL. Finally, you’ll learn how to load different types of data into pandas DataFrames.