O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Mastering Geospatial Analysis with Python

Book Description

Explore GIS processing and learn to work with various tools and libraries in Python.

About This Book
  • Analyze and process geospatial data using Python libraries such as; Anaconda, GeoPandas
  • Leverage new ArcGIS API to process geospatial data for the cloud.
  • Explore various Python geospatial web and machine learning frameworks.
Who This Book Is For

The audience for this book includes students, developers, and geospatial professionals who need a reference book that covers GIS data management, analysis, and automation techniques with code libraries built in Python 3.

What You Will Learn
  • Manage code libraries and abstract geospatial analysis techniques using Python 3.
  • Explore popular code libraries that perform specific tasks for geospatial analysis.
  • Utilize code libraries for data conversion, data management, web maps, and REST API creation.
  • Learn techniques related to processing geospatial data in the cloud.
  • Leverage features of Python 3 with geospatial databases such as PostGIS, SQL Server, and SpatiaLite.
In Detail

Python comes with a host of open source libraries and tools that help you work on professional geoprocessing tasks without investing in expensive tools. This book will introduce Python developers, both new and experienced, to a variety of new code libraries that have been developed to perform geospatial analysis, statistical analysis, and data management. This book will use examples and code snippets that will help explain how Python 3 differs from Python 2, and how these new code libraries can be used to solve age-old problems in geospatial analysis.

You will begin by understanding what geoprocessing is and explore the tools and libraries that Python 3 offers. You will then learn to use Python code libraries to read and write geospatial data. You will then learn to perform geospatial queries within databases and learn PyQGIS to automate analysis within the QGIS mapping suite. Moving forward, you will explore the newly released ArcGIS API for Python and ArcGIS Online to perform geospatial analysis and create ArcGIS Online web maps. Further, you will deep dive into Python Geospatial web frameworks and learn to create a geospatial REST API.

Style and approach

The book takes a practical, example-driven approach to teach you GIS analysis and automation techniques with Python 3.

Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Title Page
  2. Copyright and Credits
    1. Mastering Geospatial Analysis with Python
  3. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  4. Contributors
    1. About the authors
    2. About the reviewer
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Package Installation and Management
    1. Introducing Anaconda
      1. Installing Python using Anaconda
    2. Running a Jupyter Notebook
      1. Running a Notebook
        1. Creating a new Notebook
        2. Adding code
    3. Managing Python packages
      1. Managing packages with Anaconda Navigator
      2. Online searching for packages using Anaconda Cloud
      3. Managing Python packages with conda
      4. Managing Python packages using pip
      5. Upgrading and uninstalling the package with pip
    4. Python virtual environments
      1. Virtual environments using Anaconda
      2. Managing environments with conda 
      3. Virtual environments using virtualenv
    5. Summary
  7. Introduction to Geospatial Code Libraries
    1. Geospatial Data Abstraction Library (GDAL) and the OGR Simple Features Library
      1. Installing GDAL
        1. Installing GDAL using Anaconda3
        2. Installing GDAL using conda
        3. Installing GDAL using pip
        4. Installing a second GDAL version using pip
        5. Other recommended GDAL resources
    2. GEOS
      1. Installing GEOS
    3. Shapely
      1. Installing Shapely
    4. Fiona
      1. Installing Fiona
    5. Python shapefile library (pyshp)
      1. Installing pyshp
    6. pyproj
      1. Installing pyproj
    7. Rasterio
      1. Rasterio dependencies
      2. Installation of Rasterio
    8. GeoPandas
      1. GeoPandas installation
      2. GeoPandas dependencies
    9. How it all works together
    10. Summary
  8. Introduction to Geospatial Databases
    1. Installing PostgreSQL and PostGIS on Windows
    2. Installing PostgreSQL and PostGIS on Mac
    3. Working with PostgreSQL and PostGIS using Python
      1. Connecting to PostgreSQL using psycopg2
        1. Installing psycopg2
        2. Connecting to the database and creating a table
        3. Adding data to the table
        4. Shapely
        5. Querying the data
        6. Changing the CRS
        7. Buffer
        8. Distance and near
        9. Lines in the database
        10. Length of a line
        11. Intersecting lines
        12. Polygons in the database
        13. Point in polygon
    4. Summary
  9. Data Types, Storage, and Conversion
    1. Raster and vector data
      1. Shapefiles
      2. GeoJSON
      3. KML
      4. GeoPackage
    2. Raster data formats
    3. Reading and writing vector data with GeoPandas
    4. Reading and writing vector data with OGR
    5. Reading and writing raster data with Rasterio
    6. Reading and writing raster data using GDAL
    7. Summary
  10. Vector Data Analysis
    1. OGR Simple Features Library
      1. OGR batch commands
      2. ogrmerge
      3. The OGR library and Python bindings
      4. OGR's main modules and classes
        1. Creating polygon geometry with OGR
        2. Creating polygon geometry from GeoJSON
        3. Basic geometric operations
        4. Writing polygon data to a newly created shapefile
        5. Using a spatial filter to select features
    2. Shapely and Fiona
      1. Shapely objects and classes
      2. Shapely methods for geospatial analysis
      3. Fiona's data model
        1. Creating geometries with Shapely
        2. Applying geometrical methods with Shapely
        3. Reading JSON geometries with Shapely
        4. Reading data with Fiona
        5. Accessing vector geometry in shapefiles using Shapely and Fiona
    3. GeoPandas
      1. Geospatial analysis with GeoPandas
        1. Selecting and plotting geometry data with GeoPandas and Matplotlib
        2. Mapping wildfire data with GeoPandas
      2. Why data inspection matters
    4. Summary
  11. Raster Data Processing
    1. Raster operations using GDAL
      1. Using the GDAL library to load and query rasters
      2. Using GDAL to create rasters
    2. Raster operations using PostgreSQL
      1. Loading rasters into PostgreSQL
      2. Performing queries on rasters using PostgreSQL
        1. Querying raster metadata
        2. Queries returning geometry
        3. Queries returning values
    3. Summary
  12. Geoprocessing with Geodatabases
    1. A crime dashboard
      1. Building a crime database
        1. Creating the tables
        2. Populating the data
        3. Mapping queries
          1. Incidents by date
          2. Incidents in a polygon
          3. Buffers
          4. Nearest neighbor
        4. Interactive widgets
        5. Charts
        6. Triggers
    2. Summary
  13. Automating QGIS Analysis
    1. Working in the Python console
      1. Loading layers
      2. Processing a layer
        1. Layer properties
        2. Feature properties
      3. Drawing a layer from PostGIS
        1. Drawing points
        2. Drawing polygons from PostGIS
      4. Adding, editing, and deleting features
        1. Adding features to an existing layer
        2. Deleting features from an existing layer
        3. Editing features from an existing layer
      5. Selecting features using expressions
      6. Using toolboxes in Python
    2. Writing custom toolboxes
    3. Summary
  14. ArcGIS API for Python and ArcGIS Online
    1. Introducing the ArcGIS API for Python and ArcGIS Online
      1. A Pythonic web API
      2. Installing the API
        1. Testing the API
        2. Troubleshooting
      3. Authenticating your Esri user accounts
        1. Different Esri user accounts
      4. Different modules of the ArcGIS API for Python
      5. Exercise 1 – importing the API and using the map widget
        1. Creating a personalized ArcGIS Online account
      6. Exercise 2 – searching, displaying, and describing geospatial content
      7. Exercise 3 – working with raster data and the API's geoprocessing functions
    2. Summary
  15. Geoprocessing with a GPU Database
    1. Cloud geodatabase solutions
      1. Big data processing
      2. MapD architecture
        1. Cloud versus local versus combined
      3. Creating a MapD instance in the cloud
        1. Finding the AMI
        2. Opening an AWS account
        3. Creating a key pair
        4. Launching an instance
        5. Picking a version
        6. Searching for an instance
        7. Setting up a security group
      4. Immerse environment
        1. Logging in to Immerse
          1. Default dashboards
          2. NYC taxi dataset
        2. Importing a CSV
        3. Creating a chart
        4. Selections with the SQL EDITOR
        5. Use geospatial data
      5. Connecting to the database using a terminal
        1. PuTTYgen
          1. Connection configuration
          2. Using the private key
        2. Installing pymapd
          1. The conda install command
          2. The pip install command
        3. Creating a connection
          1. User and password
        4. Data cursor
        5. Creating a table
        6. Insert statements
        7. Using Apache Arrow to load data
        8. Contains queries
          1. Other available spatial SQL commands
    2. Summary
  16. Flask and GeoAlchemy2
    1. Flask and its component modules
      1. Setup
        1. Installing modules using pip
          1. Installing Flask using pip
          2. Installing Flask-SQLAlchemy via pip
          3. Installing GeoAlchemy2 using pip
          4. Installing Flask-WTForms and WTForms using pip
          5. Installing psycopg2 using pip
          6. Installing SQLAlchemy-Utils using pip
          7. Installing pyshapefile (or pyshp) using pip
          8. Installing pygeoif using pip
      2. Writing a Flask application
      3. Downloading the data from a data source
        1. County, district, state, and arena shapefiles
      4. Creating the database and data tables
        1. Adding the PostGIS extension tables to the new database
        2. Defining the database tables
          1. The declarative base
          2. Database table model classes 
          3. Table properties
        3. Creating the tables
      5. Inserting data into the new data tables
        1. Importing the required modules
        2. Locating and reading the shapefiles
          1. Accessing shapefile data
          2. Using queries
      6. Components of the Flask application
        1. Folder structure and the controller object
        2. Models
        3. Forms
          1. Dissecting the view
          2. Using forms
          3. Evaluating the request method
          4. POST requests
          5. Spatial queries
          6. Relationship query
        5. The web map template
        6. Running the web application locally
    2. Summary
  17. GeoDjango
    1. Installing and configuring Django and GeoDjango
      1. Steps from Django to GeoDjango
        1. Installing Django
          1. Installing PostGIS and psycopg2
          2. Creating the database
          3. GDAL/OGR
        2. Modifying Windows environment variables
      2. Creating a project and application
        1. Command-line argument – startproject
          1. What is created by startproject?
        2. Creating an application using manage.py
          1. What is created by manage.py
        3. Configuring settings.py
          1. Adding a new database connection
          2. Adding new installed apps
    2. Creating the application
      1. manage.py
        1. Generating data models
          1. Multipolygons 
        2. Database migrations
          1. makemigrations 
          2. sqlmigrate
          3. migrate
        3. LayerMapping
          1. Running the layer mapping 
      2. Administrative panel
        1. GeoDjango administrative panel
          1. admin.py
          2. createsuperuser
          3. runserver
      3. URLs
        1. URL patterns
        1. Required folders and files
          1. forms.py
          2. templates folder
        2. Writing views
          1. index view
          2. queryarena function
          3. arena view
      5. Running the application
    3. Summary
  18. Geospatial REST API
    1. Writing a REST API in Python
      1. REST
      2. JSON
      3. Python for REST API
        1. Flask
          1. REST modules
        2. Other frameworks
      4. Variables in Flask URLs
        1. Number converters
          1. Other data converters
      5. Request methods
        1. GET
        2. POST
        3. Other available request methods
          1. PUT
          2. DELETE
    2. The REST API application
      1. Application components
        1. Application folder and file structure
          1. app.py
          2. __init__.py
          3. The database
          4. models.py
          5. Importing required modules
          6. Declaring the session
          7. Declaring the models
        2. forms.py
        3. views.py
          1. Importing modules
        4. Base URL
        5. Arenas
          1. Getting all arenas
          2. Getting arenas by ID
          3. Getting arenas by name
          4. A geospatial query
        6. States
          1. Getting all states
          2. Getting a state by ID
          3. Getting a state by name
          4. Getting arenas by state
        7. Counties
          1. Getting a county by ID
          2. Getting a county by name
        8. Districts
          1. Getting all districts
          2. Getting a district by ID
          3. Getting a district by name
      2. API POST endpoints  
        1. New arenas
          1. The view function
          2. The addarena.html head
          3. The addarena.html script
          4. The addarena.html form
        2. Sending a POST request using the requests library
        3. Deleting an arena
      3. Running the REST API locally
    3. Deploying Flask to IIS
      1. Flask and web servers
        1. WSGI
        2. Installing the WFastCGI module and FastCGI
        3. Configuring FastCGI
        4. Root server settings and Environment Variables
    4. Summary
  19. Cloud Geodatabase Analysis and Visualization
    1. How to install CARTOframes
      1. Additional resources
      2. Jupyter Notebooks
      3. The CARTO API key
      4. Package dependencies
      5. The CARTO Data Observatory
    2. Signing up for a CARTO account
      1. A free trial of CARTO
        1. Adding a dataset
        2. The API key
        3. Adding a dataset
    3. Virtual environments
      1. Installing virtualenv
        1. Running virtualenv
      2. Activating the virtual environment
      3. Installing modules in virtualenv
        1. Modules to use
    4. Using Jupyter Notebook
      1. Connecting to an account
        1. Saving credentials
      2. Accessing a dataset
        1. Selecting individual rows
      3. Loading a CSV dataset
      4. Loading a shapefile
        1. Installing GeoPandas
        2. Writing to CARTO
      5. Loading CSV with geometry
      6. Geospatial analysis
      7. Editing and updating datasets
        1. overwrite=True
      8. Creating a map
    5. Summary
  20. Automating Cloud Cartography
    1. All things cartographic
      1. How to integrate Mapbox into your GIS 
      2. Mapbox tools
        1. MapboxGL.js
        2. Mapbox Python SDK
          1. Installing the Python SDK
      3. Getting started with Mapbox
        1. Signing up for a Mapbox account
        2. Creating an API token
        3. Adding data to a Mapbox account
          1. Tilesets
          2. Datasets
        4. Example – uploading a GeoJSON dataset
        5. Example – uploading data as a tileset
    2. Mapbox Studio
      1. Customizing a basemap
        1. Adding a tileset
      2. Virtual environment
        1. Installing MapboxGL – Jupyter
        2. Installing Jupyter Notebooks
        3. Installing Pandas and GeoPandas
      3. Using the Jupyter Notebook server
        1. Importing data using GeoPandas
        2. Creating point data from polygons
        3. Data cleanup 
        4. Saving the points as GeoJSON
        5. Adding the points to a map
        6. Creating a graduated color visualization
        7. Automatically setting colors, sizes, and breaks
      4. Creating a choropleth map
        1. Saving the map
        2. Creating a heat map
      5. Uploading data using the Mapbox Python SDK
        1. Creating the dataset
        2. Loading the data into the dataset
        3. Reading data from a dataset
        4. Deleting a row
    3. Summary
  21. Python Geoprocessing with Hadoop
    1. What is Hadoop?
    2. Installing the Hadoop framework
      1. Installing Linux
      2. Installing Docker
      3. Install Hortonworks
    3. Hadoop basics
      1. Connecting via Secure Shell
      2. Ambari
    4. Esri GIS tools for Hadoop
    5. HDFS and Hive in Python
    6. Summary
  22. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think