Python for Secret Agents

Table of contents

1. Python for Secret Agents
   1. Table of Contents
   2. Python for Secret Agents
   3. Credits
   4. About the Author
   5. About the Reviewers
   6. www.PacktPub.com
      1. Support files, eBooks, discount offers and more
         1. Why Subscribe?
         2. Free Access for Packt account holders
   7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
         1. Downloading the example code
         2. Errata
         3. Piracy
         4. Questions
   8. 1. Our Espionage Toolkit
      1. Getting the tools of the trade – Python 3.3
         1. Windows secrets
         2. Mac OS X secrets
      2. Getting more tools – a text editor
         1. Getting other developer tools
         2. Getting a tool to get more Python components
      3. Confirming our tools
         1. How do we stop?
         2. Using the help() system
            1. Mac OS and GNU/Linux secrets
            2. Windows secrets
         3. Using the help mode
      4. Background briefing – math and numbers
         1. The usual culprits
         2. The ivory tower of numbers
            1. Integer numbers
            2. Rational numbers
            3. Floating-point numbers
            4. Decimal numbers
            5. Complex numbers
         3. Outside the numbers
         4. Assigning values to variables
         5. Writing scripts and seeing output
         6. Gathering user input
            1. Handling exceptions
            2. Looping and trying again
      5. Handling text and strings
         1. Converting between numbers and strings
         2. Parsing strings
      6. Organizing our software
      7. Working with files and folders
         1. Creating a file
         2. Reading a file
         3. Defining more complex logical conditions
      8. Solving problems – recovering a lost password
         1. Reading a word corpus
         2. Reading a ZIP archive
         3. Using brute-force search
      9. Summary
   9. 2. Acquiring Intelligence Data
      1. Accessing data from the Internet
         1. Background briefing – the TCP/IP protocols
         2. Using http.client for HTTP GET
         3. Changing our client information
         4. Using FTP in Python
            1. Downloading a file via FTP
            2. Using our FTP get() function
         5. Using urllib for HTTP, FTP, or file access
         6. Using urllib for FTP access
      2. Using a REST API in Python
         1. Getting simple REST data
         2. Using more complex RESTful queries
         3. Saving our data via JSON
      3. Organizing collections of data
         1. Using a Python list
            1. Using list index operations
         2. Using a Python tuple
            1. Using generator expressions with list of tuples
         3. Using a Python dictionary mapping
            1. Using the dictionary access methods
         4. Transforming sequences with generator functions
         5. Using the defaultdict and counter mappings
         6. Using a Python set
         7. Using the for statement with a collection
         8. Using Python operators on collections
      4. Solving problems – currency conversion rates
      5. Summary
   10. 3. Encoding Secret Messages with Steganography
       1. Background briefing – handling file formats
          1. Working with the OS filesystem
             1. glob
             2. os
          2. Processing simple text files
          3. Working with ZIP files
          4. Working with JSON files
          5. Working with CSV files
          6. JPEG and PNG graphics – pixels and metadata
       2. Using the Pillow library
          1. Adding the required supporting libraries
             1. GNU/Linux secrets
             2. Mac OS X secrets
             3. Windows secrets
          2. Installing and confirming Pillow
          3. Decoding and encoding image data
          4. Manipulating images – resizing and thumbnails
          5. Manipulating images – cropping
          6. Manipulating images – enhancing
          7. Manipulating images – filtering
          8. Manipulating images – ImageOps
       3. Some approaches to steganography
          1. Getting the red-channel data
          2. Extracting bytes from Unicode characters
          3. Manipulating bits and bytes
          4. Assembling the bits
          5. Encoding the message
          6. Decoding a message
       4. Detecting and preventing tampering
          1. Using hash totals to validate a file
          2. Using a key with a digest
       5. Solving problems – encrypting a message
          1. Unpacking a message
       6. Summary
   11. 4. Drops, Hideouts, Meetups, and Lairs
       1. Background briefing – latitude, longitude, and GPS
          1. Coping with GPS device limitations
          2. Handling politics – borders, precincts, jurisdictions, and neighborhoods
       2. Finding out where we are with geocoding services
          1. Geocoding an address
          2. Reverse geocoding a latitude-longitude point
       3. How close? What direction?
          1. Combining geocoding and haversine
       4. Compressing data to make grid codes
          1. Creating GeoRef codes
       5. Decoding a GeoRef code
          1. Creating Maidenhead grid codes
          2. Decoding the Maidenhead grid codes
       6. Creating natural area codes
          1. Decoding natural area codes
       7. Solving problems – closest good restaurant
          1. Creating simple Python objects
          2. Working with HTML web services – tools
          3. Working with HTML web services – getting the page
          4. Working with HTML web services – parsing a table
          5. Making a simple Python object from columns of data
          6. Enriching Python objects with geocodes
          7. Enriching Python objects with health scores
          8. Combining the pieces and parts
          9. Working with clean data portals
          10. Making a simple Python object from a JSON document
          11. Combining different pieces and parts
          12. Final steps
          13. Understanding the data – schema and metadata
       8. Summary
   12. 5. A Spymaster's More Sensitive Analyses
       1. Creating statistical summaries
          1. Parsing the raw data file
          2. Finding an average value
             1. Understanding generator expressions
          3. Finding the value in the middle
          4. Finding the most popular value
       2. Creating Python modules and applications
          1. Creating and using a module
          2. Creating an application module
          3. Creating a hybrid module
       3. Creating our own classes of objects
          1. Using a class definition
       4. Comparisons and correlations
          1. Computing the standard deviation
          2. Computing a standardized score
             1. Comparing a sequence and an iterable
          3. Computing a coefficient of correlation
       5. Writing high-quality software
          1. Building a self-testing module and a test module
          2. Creating more sophisticated tests
          3. Adding doctest cases to a class definition
       6. Solving problems – analyzing some interesting datasets
          1. Getting some more data
          2. Further research
       7. Summary
   13. Index