Learning Python Networking - Second Edition

Learning Python Networking - Second Edition

by Jose Manuel Ortega, Dr. M. O. Faruque Sarker, Sam Washington
Released March 2019
Publisher(s): Packt Publishing
ISBN: 9781789958096

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Achieve improved network programmability and automation by leveraging powerful network programming concepts, algorithms, and tools

Key Features

  • Deal with remote network servers using SSH, FTP, SNMP and LDAP protocols.
  • Design multi threaded and event-driven architectures for asynchronous servers programming.
  • Leverage your Python programming skills to build powerful network applications

Book Description

Network programming has always been a demanding task. With full-featured and well-documented libraries all the way up the stack, Python makes network programming the enjoyable experience it should be.

Starting with a walk through of today's major networking protocols, through this book, you'll learn how to employ Python for network programming, how to request and retrieve web resources, and how to extract data in major formats over the web. You will utilize Python for emailing using different protocols, and you'll interact with remote systems and IP and DNS networking. You will cover the connection of networking devices and configuration using Python 3.7, along with cloud-based network management tasks using Python.

As the book progresses, socket programming will be covered, followed by how to design servers, and the pros and cons of multithreaded and event-driven architectures. You'll develop practical clientside applications, including web API clients, email clients, SSH, and FTP. These applications will also be implemented through existing web application frameworks.

What you will learn

  • Execute Python modules on networking tools
  • Automate tasks regarding the analysis and extraction of information from a network
  • Get to grips with asynchronous programming modules available in Python
  • Get to grips with IP address manipulation modules using Python programming
  • Understand the main frameworks available in Python that are focused on web application
  • Manipulate IP addresses and perform CIDR calculations

Who this book is for

If you're a Python developer or a system administrator with Python experience and you're looking to take your first steps in network programming, then this book is for you. If you're a network engineer or a network professional aiming to be more productive and efficient in networking programmability and automation then this book would serve as a useful resource. Basic knowledge of Python is assumed.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Learning Python Networking Second Edition
  3. About Packt
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the authors
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Section 1: Introduction to Network and HTTP Programming
  7. Network Programming with Python
    1. Technical requirements
    2. An introduction to TCP/IP networking
      1. Introduction to TCP/IP
      2. The protocol stack, layer by layer
      3. UDP
      4. TCP
    3. Protocol concepts and the problems that protocols solve
      1. IP addresses and ports
      2. Network interfaces
      3. UDP versus TCP
      4. DHCP
      5. DNS
    4. Addressing
      1. NAT
      2. IPv4
      3. IPv6
    5. Python network programming through libraries
      1. An introduction to the PyPI Python repository
      2. Alternatives to pip for installing packages
        1. Conda
        2. Virtualenv
        3. Pipenv
      3. An introduction to libraries for network programming with Python
        1. Introduction to sockets
        2. Socket module in Python
        3. Client socket methods
        4. Server socket methods
      4. Working with RFC
        1. Extracting RFC information
        2. Downloading an RFC with urllib
        3. Downloading an RFC with requests
        4. Downloading an RFC with the socket module
    6. Interacting with Wireshark with pyshark
      1. Introduction to Wireshark
        1. Wireshark installation
      2. Capturing packets with Wireshark
        1. Network traffic in Wireshark
        2. Color coding in Wireshark
        3. Working with filters in Wireshark
        4. Filtering by protocol name
        5. HTTP objects filter
        6. Capture filters
        7. Display filters
      3. Analyzing networking traffic using the pyshark library
      4. FileCapture and LiveCapture in pyshark
    7. Summary
    8. Questions
    9. Further reading
  8. Programming for the Web with HTTP
    1. Technical requirements
    2. Consuming web services in Python with urllib
      1. Status codes
      2. Handling exceptions
      3. HTTP headers
      4. User agent
      5. Customizing requests with urllib
      6. Getting headers with a proxy
      7. Content types
      8. Extracting links from a URL with urllib
      9. Getting images from a URL with urllib
      10. Working with URLs
    3. Consuming web services in Python with requests
      1. Introduction to requests
      2. Checking HTTP headers
      3. Proxy requests
      4. Get whois information
      5. Working with JSON
    4. Handling forms with urllib and requests with Python 3.7
      1. Handling forms with urllib
      2. Handling forms with requests
    5. Handling cookies with urllib and requests with Python
      1. What are cookies?
      2. Handling cookies with urllib
      3. Cookie handling with requests
    6. Handling HTTP Basic and Digest Authentication with requests
      1. Introduction to authentication mechanisms
      2. HTTP Basic authentication
      3. HTTP Digest authentication
    7. Summary
    8. Questions
    9. Further reading
  9. Section 2: Interacting with APIs, Web Scraping, and Server Scripting
  10. Application Programming Interface in Action
    1. Technical requirements
    2. Introduction to REST APIs
      1. Advantages of using REST APIs
    3. Introduction to JSON and the JSON module
      1. Encoding and decoding with the JSON package
      2. Using dict with JSON
    4. Interacting with a JSON hybrid-REST API (Twitter)
      1. The Twitter API
      2. Registering your application for the Twitter API
      3. Authenticating requests with OAuth
      4. Collecting information from Twitter
        1. A Twitter client
        2. Retrieving tweets from a timeline
        3. Searching tweets
      5. Consuming the Twitter REST API with Python
      6. Connecting with the Twitter API
      7. Accessing Twitter API resources
      8. Streaming APIs with Tweepy
    5. Introduction to XML
      1. Getting started with XML
      2. The XML APIs
        1. Processing XML with ElementTree
        2. Pretty printing
      3. Reading an XML file
    6. Working with XML and a full REST API (Amazon S3 bucket) with the Boto module
      1. The Amazon S3 API
      2. Registering with AWS
      3. Authentication with AWS
      4. S3 buckets and objects
      5. Creating a bucket with the S3 API
      6. Uploading and downloading file
      7. Listing buckets
      8. Parsing XML and handling errors
      9. Connecting to S3 with the Python Boto package
    7. Summary
    8. Questions
    9. Further reading
  11. Web Scraping with BeautifulSoup and Scrapy
    1. Technical requirements
    2. Introduction to web scraping
      1. Web content extraction
      2. What is web scraping?
      3. HTML parsers
      4. Parsing HTML with lxml
      5. Searching with XPath
    3. Extracting information from web pages and parsing HTML with BeautifulSoup
      1. BeautifulSoup introduction
      2. Access to elements through DOM
      3. Extracting labels using regex
      4. Handling URL exceptions and not found tags
    4. Introduction to Scrapy components and architecture
      1. What is Scrapy?
      2. Scrapy architecture
      3. XPath expressions
    5. Scrapy as a framework for performing web crawling processes and data analysis
      1. Installation of Scrapy
      2. Creating a project with Scrapy
      3. Scrapy item class
      4. Spiders
        1. Creating our spider
      5. Pipelines items and export formats
      6. Scrapy settings
      7. Executing Scrapy
        1. Scrapy execution tips and tricks
      8. EuroPython project
        1. Executing EuroPython spider
    6. Working with Scrapy in the cloud
      1. Scrapinghub
      2. Portia
        1. Start pages and link crawling
    7. Summary
    8. Questions
    9. Further reading
  12. Engaging with Email
    1. Technical requirements
    2. Introduction to email protocols
    3. Sending emails with SMTP through the smtplib library
      1. SMTP protocol
      2. Working with smtplib
        1. Sending a basic message
        2. Sending messages in HTML format
        3. Sending emails to multiple recipients
        4. Sending an email with attachments
      3. Authentication with TLS
        1. Establishing a connection with a Gmail SMTP server
      4. Using an external SMTP service
        1. Creating and sending an email with an attachment
    4. Learning the POP3 protocol and retrieving emails with poplib
      1. Understanding the POP3 protocol
      2. Introduction to poplib
      3. Retrieving emails with SSL
      4. Establishing a connection with Gmail for reading emails
        1. Gmail account configuration
        2. Unread messages
    5. Manipulating and retrieving emails on the server email using IMAP with imapclient and imaplib
      1. IMAP protocol
      2. Retrieving emails with imaplib
      3. Retrieving emails with imapclient
    6. Summary
    7. Questions
    8. Further reading
  13. Interacting with Remote Systems
    1. Technical requirements
    2. Understanding the SSH protocol
      1. SSH introduction
      2. Using SSH to encrypt sessions
      3. How the SSH protocol works
      4. SSH service features
      5. Configuring the SSH protocol to make it more secure
    3. SSH terminals and running commands with paramiko
      1. Installing paramiko
      2. Establishing an SSH connection with paramiko
      3. Running commands with paramiko
      4. Running an interactive shell with paramiko
      5. SFTP with paramiko
      6. Paramiko alternatives
        1. Fabric
    4. Understanding the FTP protocol for transferring files
      1. The File Transfer Protocol
      2. Introduction to ftplib
      3. Other ftplib functions
      4. Inspecting FTP packets with Wireshark
    5. Reading and interacting with SNMP servers
      1. The SNMP
      2. MIB – a broad base of information
      3. Introduction to pysnmp
      4. Polling information from the SNMP agent
    6. Reading and interacting with LDAP servers
      1. The LDAP protocol
      2. LDAP terminology
      3. Introduction to python-ldap
      4. The LDAP FreeIPA server
      5. Working with LDAP3
        1. Accessing the LDAP server
      6. Finding entries in LDAP
    7. Summary
    8. Questions
    9. Further reading
  14. Section 3: IP Address Manipulation and Network Automation
  15. Working with IP and DNS
    1. Technical requirements
    2. Principles of the IP protocol
      1. Resolving the IP address with the socket package
      2. Validating the IP address with the socket package
    3. Retrieving the network configuration of a local machine
      1. Gathering information with the netifaces package
    4. Using Python to manipulate IP addresses and perform CIDR calculations
      1. The Python ipaddress module
      2. Manipulating IP addresses
        1. IP network objects
      3. Subnetting in Python
      4. Network interface objects
        1. IP address objects
      5. Planning IP addresses for your local area network
    5. The dnspython module as a tool for extracting information from DNS servers
      1. Working with dnspython
        1. Determining the destination of an MX record and its preference
        2. Manipulating domain names
        3. Converting IPv4 and IPv6 addresses into their DNS reverse map names
      2. Inspecting the DNS client and server communication
    6. GeoIP lookups with pygeoip and python-geoip
      1. Introduction to geolocation
      2. Introduction to pygeoip
      3. Introduction to python-geoip
      4. The MaxMind database in Python
    7. Summary
    8. Questions
    9. Further reading
  16. Implementing IPv6 and Address Manipulation
    1. Technical requirements
    2. Learning and understanding the IPv6 protocol
      1. The IPv6 protocol
      2. IPv6 addresses
      3. Representation of IPv6 addresses
      4. Reserved IPv6 addresses
      5. First steps with IPv6 – link-local
    3. Create an echo client and server with IPv6
      1. Working with sockets
        1. The socket server
      2. The socket client
      3. Executing client and server
    4. Understanding netifaces module for checking IPv6 support on your network
      1. Introduction to netifaces
      2. Other packages for getting interfaces
    5. Using the netaddr module as a network-address manipulation library for Python
      1. Operating with IPv6
    6. Understand ipaddress module as IPv4 and IPv6 manipulation library
      1. The Python ipaddress module
      2. IP network objects
      3. Subnetting in Python with IPv6
      4. Network interface objects
      5. The IP address objects
      6. Planning IP addresses for your local area network
    7. Summary
    8. Questions
    9. Further reading
  17. Performing Network Automation with Python and Ansible
    1. Technical requirements
    2. Basics of Ansible
      1. Ansible introduction
      2. Installing Ansible
      3. Configuring Ansible
      4. Using Ansible
    3. Ansible's components and architecture
      1. Ansible's architecture
      2. Ansible's inventory file
    4. Automating network Python tasks with Ansible
      1. Ansible tasks
      2. Ad-hoc commands
      3. Using playbooks
    5. Writing Ansible modules with Python
      1. Introduction to Ansible modules
      2. Implementing Ansible modules with Python
    6. Summary
    7. Questions
    8. Further reading
  18. Section 4: Sockets and Server Programming
  19. Programming with Sockets
    1. Technical requirements
    2. Basics of sockets
      1. Sockets introduction
      2. Socket types
      3. Getting information about ports, protocols, and domains
      4. Creating a TCP client
      5. Banner grabbing with the socket module
      6. Port scanning with sockets
      7. Inspecting the client and server communication
    3. Working with UDP and TCP sockets in Python 3.7
      1. Introduction to the TCP and UDP protocols
      2. Starting network programming with Python
      3. TCP sockets
        1. Starting a client
      4. Capturing packets in a loopback interface
      5. Inspecting the client and server interaction
      6. Code limitations
      7. Creating a simple UDP client and UDP server
        1. Implementing the UDP server
        2. Implementing the UDP client
    4. Working with IPv6 sockets in Python 3.7
      1. Implementing the IPv6 server
      2. Implementing the IPv6 client
      3. Executing client and server
    5. Non-blocking and asynchronous socket I/O
      1. Introducing non-blocking I/O
      2. The client-server model with multiple connections
    6. HTTPS and securing sockets with TLS
      1. Implementing the SSL client
      2. Inspecting standard SSL client and server communication
    7. Summary
    8. Questions
    9. Further reading
  20. Designing Servers and Asynchronous Programming
    1. Technical requirements
    2. Building a multiprocessing-based TCP server
      1. Introducing the concurrent.futures module
      2. Application for checking websites
      3. The multiprocessing approach
    3. Building asynchronous applications with asyncio and aiohttp
      1. Introducing asyncio
      2. Using asyncio
      3. Introducing event loops
      4. Futures
      5. Task manipulation with asyncio
      6. Downloading files with asyncio
      7. Introducing aiohttp
      8. Downloading files with aiohttp
      9. Other event loop solutions
    4. Building asynchronous network applications with Tornado
      1. Introducing Tornado
      2. Implementing the Tornado web server
      3. Implementing an asynchronous client with AsyncHTTPClient
      4. Asynchronous generators
      5. Utilities in Tornado for asynchronous network operations
    5. Building asynchronous network applications with Twisted
      1. Introduction to Twisted
      2. Protocols
      3. Building a basic Twisted server
      4. Factory
      5. Reactor
      6. Building a socket client
      7. Executing the client and server
      8. Building a Twisted client
      9. Building a Twisted web server
    6. Building asynchronous network applications with Celery
      1. Celery architecture
      2. Installing Celery
      3. Installing Redis
      4. Introduction to Redis
      5. Distributing Python with Celery and Redis
    7. Summary
    8. Questions
    9. Further reading
  21. Designing Applications on the Web
    1. Technical requirements
    2. Writing a web application with WSGI
      1. Introducing WSGI
      2. Creating a WSGI application
    3. Existing web application frameworks (Django, Flask, and Plone)
      1. Web frameworks
    4. The MVC pattern and dynamic web programming with Python
      1. The MVC pattern
      2. Dynamic web pages
      3. Processing dynamic pages
      4. Accessing a database
      5. Django introduction
      6. Creating a Django application
    5. Creating RESTful web applications and working with Flask and HTTP requests
      1. Introducing Flask
      2. Routing in Flask
      3. Jinja2 templating
      4. POST parameters with Flask
      5. Other templating engines
      6. Flask extensions
    6. Working with a database in Flask with SQLAlchemy
      1. Introducing SQLAlchemy
      2. Creating a session and ORM queries
      3. Using Flask with SQLAlchemy
    7. Summary
    8. Questions
    9. Further reading
  22. Assessment
    1. Chapter 1, Network Programming with Python
    2. Chapter 2, Programming for the Web with HTTP
    3. Chapter 3, Application Programming Interface in Action
    4. Chapter 4, Web Scraping with BeautifulSoup and Scrapy
    5. Chapter 5, Engaging with Email
    6. Chapter 6, Interacting with Remote Systems
    7. Chapter 7, Working with IP and DNS
    8. Chapter 8, Implementing IPv6 and Address Manipulation
    9. Chapter 9, Performing Network Automation with Python and Ansible
    10. Chapter 10, Programming with Sockets
    11. Chapter 11, Designing Servers and Asynchronous Programming
    12. Chapter 12, Designing Applications on the Web
  23. Another Book You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Learning Python Networking - Second Edition
  • Author(s): Jose Manuel Ortega, Dr. M. O. Faruque Sarker, Sam Washington
  • Release date: March 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781789958096