Mastering Geospatial Analysis with Python

Table of contents

1. Title Page
2. Copyright and Credits
   1. Mastering Geospatial Analysis with Python
3. Packt Upsell
   1. Why subscribe?
   2. PacktPub.com
4. Contributors
   1. About the authors
   2. About the reviewer
   3. Packt is searching for authors like you
5. Preface
   1. Who this book is for
   2. What this book covers
   3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
   4. Get in touch
      1. Reviews
6. Package Installation and Management
   1. Introducing Anaconda
      1. Installing Python using Anaconda
   2. Running a Jupyter Notebook
      1. Running a Notebook
         1. Creating a new Notebook
         2. Adding code
   3. Managing Python packages
      1. Managing packages with Anaconda Navigator
      2. Online searching for packages using Anaconda Cloud
      3. Managing Python packages with conda
      4. Managing Python packages using pip
      5. Upgrading and uninstalling the package with pip
   4. Python virtual environments
      1. Virtual environments using Anaconda
      2. Managing environments with conda 
      3. Virtual environments using virtualenv
   5. Summary
7. Introduction to Geospatial Code Libraries
   1. Geospatial Data Abstraction Library (GDAL) and the OGR Simple Features Library
      1. Installing GDAL
         1. Installing GDAL using Anaconda3
         2. Installing GDAL using conda
         3. Installing GDAL using pip
         4. Installing a second GDAL version using pip
         5. Other recommended GDAL resources
   2. GEOS
      1. Installing GEOS
   3. Shapely
      1. Installing Shapely
   4. Fiona
      1. Installing Fiona
   5. Python shapefile library (pyshp)
      1. Installing pyshp
   6. pyproj
      1. Installing pyproj
   7. Rasterio
      1. Rasterio dependencies
      2. Installation of Rasterio
   8. GeoPandas
      1. GeoPandas installation
      2. GeoPandas dependencies
   9. How it all works together
   10. Summary
8. Introduction to Geospatial Databases
   1. Installing PostgreSQL and PostGIS on Windows
   2. Installing PostgreSQL and PostGIS on Mac
   3. Working with PostgreSQL and PostGIS using Python
      1. Connecting to PostgreSQL using psycopg2
         1. Installing psycopg2
         2. Connecting to the database and creating a table
         3. Adding data to the table
         4. Shapely
         5. Querying the data
         6. Changing the CRS
         7. Buffer
         8. Distance and near
         9. Lines in the database
         10. Length of a line
         11. Intersecting lines
         12. Polygons in the database
         13. Point in polygon
   4. Summary
9. Data Types, Storage, and Conversion
   1. Raster and vector data
      1. Shapefiles
      2. GeoJSON
      3. KML
      4. GeoPackage
   2. Raster data formats
   3. Reading and writing vector data with GeoPandas
   4. Reading and writing vector data with OGR
   5. Reading and writing raster data with Rasterio
   6. Reading and writing raster data using GDAL
   7. Summary
10. Vector Data Analysis
    1. OGR Simple Features Library
       1. OGR batch commands
       2. ogrmerge
       3. The OGR library and Python bindings
       4. OGR's main modules and classes
          1. Creating polygon geometry with OGR
          2. Creating polygon geometry from GeoJSON
          3. Basic geometric operations
          4. Writing polygon data to a newly created shapefile
          5. Using a spatial filter to select features
    2. Shapely and Fiona
       1. Shapely objects and classes
       2. Shapely methods for geospatial analysis
       3. Fiona's data model
          1. Creating geometries with Shapely
          2. Applying geometrical methods with Shapely
          3. Reading JSON geometries with Shapely
          4. Reading data with Fiona
          5. Accessing vector geometry in shapefiles using Shapely and Fiona
    3. GeoPandas
       1. Geospatial analysis with GeoPandas
          1. Selecting and plotting geometry data with GeoPandas and Matplotlib
          2. Mapping wildfire data with GeoPandas
       2. Why data inspection matters
    4. Summary
11. Raster Data Processing
    1. Raster operations using GDAL
       1. Using the GDAL library to load and query rasters
       2. Using GDAL to create rasters
    2. Raster operations using PostgreSQL
       1. Loading rasters into PostgreSQL
       2. Performing queries on rasters using PostgreSQL
          1. Querying raster metadata
          2. Queries returning geometry
          3. Queries returning values
    3. Summary
12. Geoprocessing with Geodatabases
    1. A crime dashboard
       1. Building a crime database
          1. Creating the tables
          2. Populating the data
          3. Mapping queries
             1. Incidents by date
             2. Incidents in a polygon
             3. Buffers
             4. Nearest neighbor
          4. Interactive widgets
          5. Charts
          6. Triggers
    2. Summary
13. Automating QGIS Analysis
    1. Working in the Python console
       1. Loading layers
       2. Processing a layer
          1. Layer properties
          2. Feature properties
       3. Drawing a layer from PostGIS
          1. Drawing points
          2. Drawing polygons from PostGIS
       4. Adding, editing, and deleting features
          1. Adding features to an existing layer
          2. Deleting features from an existing layer
          3. Editing features from an existing layer
       5. Selecting features using expressions
       6. Using toolboxes in Python
    2. Writing custom toolboxes
    3. Summary
14. ArcGIS API for Python and ArcGIS Online
    1. Introducing the ArcGIS API for Python and ArcGIS Online
       1. A Pythonic web API
       2. Installing the API
          1. Testing the API
          2. Troubleshooting
       3. Authenticating your Esri user accounts
          1. Different Esri user accounts
       4. Different modules of the ArcGIS API for Python
       5. Exercise 1 – importing the API and using the map widget
          1. Creating a personalized ArcGIS Online account
       6. Exercise 2 – searching, displaying, and describing geospatial content
       7. Exercise 3 – working with raster data and the API's geoprocessing functions
    2. Summary
15. Geoprocessing with a GPU Database
    1. Cloud geodatabase solutions
       1. Big data processing
       2. MapD architecture
          1. Cloud versus local versus combined
       3. Creating a MapD instance in the cloud
          1. Finding the AMI
          2. Opening an AWS account
          3. Creating a key pair
          4. Launching an instance
          5. Picking a version
          6. Searching for an instance
          7. Setting up a security group
       4. Immerse environment
          1. Logging in to Immerse
             1. Default dashboards
             2. NYC taxi dataset
          2. Importing a CSV
          3. Creating a chart
          4. Selections with the SQL EDITOR
          5. Use geospatial data
       5. Connecting to the database using a terminal
          1. PuTTYgen
             1. Connection configuration
             2. Using the private key
          2. Installing pymapd
             1. The conda install command
             2. The pip install command
          3. Creating a connection
             1. User and password
          4. Data cursor
          5. Creating a table
          6. Insert statements
          7. Using Apache Arrow to load data
          8. Contains queries
             1. Other available spatial SQL commands
    2. Summary
16. Flask and GeoAlchemy2
    1. Flask and its component modules
       1. Setup
          1. Installing modules using pip
             1. Installing Flask using pip
             2. Installing Flask-SQLAlchemy via pip
             3. Installing GeoAlchemy2 using pip
             4. Installing Flask-WTForms and WTForms using pip
             5. Installing psycopg2 using pip
             6. Installing SQLAlchemy-Utils using pip
             7. Installing pyshapefile (or pyshp) using pip
             8. Installing pygeoif using pip
       2. Writing a Flask application
       3. Downloading the data from a data source
          1. County, district, state, and arena shapefiles
       4. Creating the database and data tables
          1. Adding the PostGIS extension tables to the new database
          2. Defining the database tables
             1. The declarative base
             2. Database table model classes 
             3. Table properties
          3. Creating the tables
       5. Inserting data into the new data tables
          1. Importing the required modules
          2. Locating and reading the shapefiles
             1. Accessing shapefile data
             2. Using queries
       6. Components of the Flask application
          1. Folder structure and the controller object
          2. Models
          3. Forms
          4. Views
             1. Dissecting the view
             2. Using forms
             3. Evaluating the request method
             4. POST requests
             5. Spatial queries
             6. Relationship query
          5. The web map template
          6. Running the web application locally
    2. Summary
17. GeoDjango
    1. Installing and configuring Django and GeoDjango
       1. Steps from Django to GeoDjango
          1. Installing Django
             1. Installing PostGIS and psycopg2
             2. Creating the database
             3. GDAL/OGR
          2. Modifying Windows environment variables
       2. Creating a project and application
          1. Command-line argument – startproject
             1. What is created by startproject?
          2. Creating an application using manage.py
             1. What is created by manage.py
          3. Configuring settings.py
             1. Adding a new database connection
             2. Adding new installed apps
    2. Creating the application
       1. manage.py
          1. Generating data models
             1. Multipolygons 
          2. Database migrations
             1. makemigrations 
             2. sqlmigrate
             3. migrate
          3. LayerMapping
             1. Running the layer mapping 
       2. Administrative panel
          1. GeoDjango administrative panel
             1. admin.py
             2. createsuperuser
             3. runserver
       3. URLs
          1. URL patterns
       4. Views
          1. Required folders and files
             1. forms.py
             2. templates folder
          2. Writing views
             1. index view
             2. queryarena function
             3. arena view
       5. Running the application
    3. Summary
18. Geospatial REST API
    1. Writing a REST API in Python
       1. REST
       2. JSON
       3. Python for REST API
          1. Flask
             1. REST modules
          2. Other frameworks
       4. Variables in Flask URLs
          1. Number converters
             1. Other data converters
       5. Request methods
          1. GET
          2. POST
          3. Other available request methods
             1. PUT
             2. DELETE
    2. The REST API application
       1. Application components
          1. Application folder and file structure
             1. app.py
             2. __init__.py
             3. The database
             4. models.py
             5. Importing required modules
             6. Declaring the session
             7. Declaring the models
          2. forms.py
          3. views.py
             1. Importing modules
          4. Base URL
          5. Arenas
             1. Getting all arenas
             2. Getting arenas by ID
             3. Getting arenas by name
             4. A geospatial query
          6. States
             1. Getting all states
             2. Getting a state by ID
             3. Getting a state by name
             4. Getting arenas by state
          7. Counties
             1. Getting a county by ID
             2. Getting a county by name
          8. Districts
             1. Getting all districts
             2. Getting a district by ID
             3. Getting a district by name
       2. API POST endpoints  
          1. New arenas
             1. The view function
             2. The addarena.html head
             3. The addarena.html script
             4. The addarena.html form
          2. Sending a POST request using the requests library
          3. Deleting an arena
       3. Running the REST API locally
    3. Deploying Flask to IIS
       1. Flask and web servers
          1. WSGI
          2. Installing the WFastCGI module and FastCGI
          3. Configuring FastCGI
          4. Root server settings and Environment Variables
    4. Summary
19. Cloud Geodatabase Analysis and Visualization
    1. How to install CARTOframes
       1. Additional resources
       2. Jupyter Notebooks
       3. The CARTO API key
       4. Package dependencies
       5. The CARTO Data Observatory
    2. Signing up for a CARTO account
       1. A free trial of CARTO
          1. Adding a dataset
          2. The API key
          3. Adding a dataset
    3. Virtual environments
       1. Installing virtualenv
          1. Running virtualenv
       2. Activating the virtual environment
       3. Installing modules in virtualenv
          1. Modules to use
    4. Using Jupyter Notebook
       1. Connecting to an account
          1. Saving credentials
       2. Accessing a dataset
          1. Selecting individual rows
       3. Loading a CSV dataset
       4. Loading a shapefile
          1. Installing GeoPandas
          2. Writing to CARTO
       5. Loading CSV with geometry
       6. Geospatial analysis
       7. Editing and updating datasets
          1. overwrite=True
       8. Creating a map
    5. Summary
20. Automating Cloud Cartography
    1. All things cartographic
       1. How to integrate Mapbox into your GIS 
       2. Mapbox tools
          1. MapboxGL.js
          2. Mapbox Python SDK
             1. Installing the Python SDK
       3. Getting started with Mapbox
          1. Signing up for a Mapbox account
          2. Creating an API token
          3. Adding data to a Mapbox account
             1. Tilesets
             2. Datasets
          4. Example – uploading a GeoJSON dataset
          5. Example – uploading data as a tileset
    2. Mapbox Studio
       1. Customizing a basemap
          1. Adding a tileset
       2. Virtual environment
          1. Installing MapboxGL – Jupyter
          2. Installing Jupyter Notebooks
          3. Installing Pandas and GeoPandas
       3. Using the Jupyter Notebook server
          1. Importing data using GeoPandas
          2. Creating point data from polygons
          3. Data cleanup 
          4. Saving the points as GeoJSON
          5. Adding the points to a map
          6. Creating a graduated color visualization
          7. Automatically setting colors, sizes, and breaks
       4. Creating a choropleth map
          1. Saving the map
          2. Creating a heat map
       5. Uploading data using the Mapbox Python SDK
          1. Creating the dataset
          2. Loading the data into the dataset
          3. Reading data from a dataset
          4. Deleting a row
    3. Summary
21. Python Geoprocessing with Hadoop
    1. What is Hadoop?
    2. Installing the Hadoop framework
       1. Installing Linux
       2. Installing Docker
       3. Install Hortonworks
    3. Hadoop basics
       1. Connecting via Secure Shell
       2. Ambari
    4. Esri GIS tools for Hadoop
    5. HDFS and Hive in Python
    6. Summary
22. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think