Effective Computation in Physics

Effective Computation in Physics

by Anthony Scopatz, Kathryn D. Huff
Released June 2015
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491901533

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to live online training experiences, plus books, videos, and digital content from O’Reilly and nearly 200 trusted publishing partners.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

More physicists today are taking on the role of software developer as part of their research, but software development isnâ??t always easy or obvious, even for physicists. This practical book teaches essential software development skills to help you automate and accomplish nearly any aspect of research in a physics-based field.

Written by two PhDs in nuclear engineering, this book includes practical examples drawn from a working knowledge of physics concepts. Youâ??ll learn how to use the Python programming language to perform everything from collecting and analyzing data to building software and publishing your results.

In four parts, this book includes:

  • Getting Started: Jump into Python, the command line, data containers, functions, flow control and logic, and classes and objects
  • Getting It Done: Learn about regular expressions, analysis and visualization, NumPy, storing data in files and HDF5, important data structures in physics, computing in parallel, and deploying software
  • Getting It Right: Build pipelines and software, learn to use local and remote version control, and debug and test your code
  • Getting It Out There: Document your code, process and publish your findings, and collaborate efficiently; dive into software licenses, ownership, and copyright procedures

Table of contents

  1. Foreword
  2. Preface
    1. What Is This Book?
    2. Who This Book Is For
    3. Who This Book Is Not For
    4. Case Study on How to Use This Book: Radioactive Decay Constants
      1. Accessing Data and Libraries
      2. Creating a Simple Program
      3. Automating Data Collection
      4. Analyzing and Plotting the Data
      5. Keeping Track of Changes
      6. Testing the Code
      7. Documenting the Code
      8. Publishing
    5. What to Do While Reading This Book
    6. Conventions Used in This Book
    7. Using Code Examples
    8. Installation and Setup
      1. Step 1: Download and Install Miniconda (or Anaconda)
      2. Step 2: Install the Packages
    9. Safari® Books Online
    10. How to Contact Us
    11. Acknowledgments
  3. I. Getting Started
  4. 1. Introduction to the Command Line
    1. Navigating the Shell
      1. The Shell Is a Programming Language
      2. Paths and pwd
      3. Home Directory (~)
      4. Listing the Contents (ls)
      5. Changing Directories (cd)
      6. File Inspection (head and tail)
    2. Manipulating Files and Directories
      1. Creating Files (nano, emacs, vi, cat, >, and touch)
      2. Copying and Renaming Files (cp and mv)
      3. Making Directories (mkdir)
      4. Deleting Files and Directories (rm)
      5. Flags and Wildcards
    3. Getting Help
      1. Reading the Manual (man)
      2. Finding the Right Hammer (apropos)
      3. Combining Utilities with Redirection and Pipes (>, >>, and |)
    4. Permissions and Sharing
      1. Seeing Permissions (ls -l)
      2. Setting Ownership (chown)
      3. Setting Permissions (chmod)
      4. Creating Links (ln)
      5. Connecting to Other Computers (ssh and scp)
    5. The Environment
      1. Saving Environment Variables (.bashrc)
      2. Running Programs (PATH)
      3. Nicknaming Commands (alias)
    6. Scripting with Bash
    7. Command Line Wrap-up
  5. 2. Programming Blastoff with Python
    1. Running Python
    2. Comments
    3. Variables
    4. Special Variables
      1. Boolean Values
      2. None Is Not Zero!
      3. NotImplemented Is Not None!
    5. Operators
    6. Strings
      1. String Indexing
      2. String Concatenation
      3. String Literals
      4. String Methods
    7. Modules
      1. Importing Modules
      2. Importing Variables from a Module
      3. Aliasing Imports
      4. Aliasing Variables on Import
      5. Packages
      6. The Standard Library and the Python Ecosystem
    8. Python Wrap-up
  6. 3. Essential Containers
    1. Lists
    2. Tuples
    3. Sets
    4. Dictionaries
    5. Containers Wrap-up
  7. 4. Flow Control and Logic
    1. Conditionals
      1. if-else Statements
      2. if-elif-else Statements
      3. if-else Expression
    2. Exceptions
      1. Raising Exceptions
    3. Loops
      1. while Loops
      2. for Loops
      3. Comprehensions
    4. Flow Control and Logic Wrap-up
  8. 5. Operating with Functions
    1. Functions in Python
    2. Keyword Arguments
    3. Variable Number of Arguments
    4. Multiple Return Values
    5. Scope
    6. Recursion
    7. Lambdas
    8. Generators
    9. Decorators
    10. Function Wrap-up
  9. 6. Classes and Objects
    1. Object Orientation
    2. Objects
    3. Classes
      1. Class Variables
      2. Instance Variables
      3. Constructors
      4. Methods
      5. Static Methods
      6. Duck Typing
      7. Polymorphism
    4. Decorators and Metaclasses
    5. Object Orientation Wrap-up
  10. II. Getting It Done
  11. 7. Analysis and Visualization
    1. Preparing Data
      1. Experimental Data
      2. Simulation Data
      3. Metadata
    2. Loading Data
      1. NumPy
      2. PyTables
      3. Pandas
      4. Blaze
    3. Cleaning and Munging Data
      1. Missing Data
    4. Analysis
      1. Model-Driven Analysis
      2. Data-Driven Analysis
    5. Visualization
      1. Visualization Tools
      2. Gnuplot
      3. matplotlib
      4. Bokeh
      5. Inkscape
    6. Analysis and Visualization Wrap-up
  12. 8. Regular Expressions
    1. Messy Magnetism
    2. Metacharacters on the Command Line
      1. Listing Files with Simple Patterns
      2. Globally Finding Filenames with Patterns (find)
    3. grep, sed, and awk
    4. Finding Patterns in Files (grep)
    5. Finding and Replacing Patterns in Files (sed)
      1. Finding and Replacing a Complex Pattern
      2. sed Extras
    6. Manipulating Columns of Data (awk)
    7. Python Regular Expressions
    8. Regular Expressions Wrap-up
  13. 9. NumPy: Thinking in Arrays
    1. Arrays
    2. dtypes
    3. Slicing and Views
    4. Arithmetic and Broadcasting
    5. Fancy Indexing
    6. Masking
    7. Structured Arrays
    8. Universal Functions
    9. Other Valuable Functions
    10. NumPy Wrap-up
  14. 10. Storing Data: Files and HDF5
    1. Files in Python
    2. An Aside About Computer Architecture
    3. Big Ideas in HDF5
    4. File Manipulations
    5. Hierarchy Layout
    6. Chunking
    7. In-Core and Out-of-Core Operations
      1. In-Core
      2. Out-of-Core
    8. Querying
    9. Compression
    10. HDF5 Utilities
    11. Storing Data Wrap-up
  15. 11. Important Data Structures in Physics
    1. Hash Tables
      1. Resizing
      2. Collisions
    2. Data Frames
      1. Series
      2. The Data Frame Structure
    3. B-Trees
    4. K-D Trees
    5. Data Structures Wrap-up
  16. 12. Performing in Parallel
    1. Scale and Scalability
    2. Problem Classification
    3. Example: N-Body Problem
    4. No Parallelism
    5. Threads
    6. Multiprocessing
    7. MPI
    8. Parallelism Wrap-up
  17. 13. Deploying Software
    1. Deploying the Software Itself
      1. pip
      2. Conda
      3. Virtual Machines
      4. Docker
    2. Deploying to the Cloud
    3. Deploying to Supercomputers
    4. Deployment Wrap-up
  18. III. Getting It Right
  19. 14. Building Pipelines and Software
    1. make
      1. Running make
      2. Makefiles
      3. Targets
      4. Special Targets
    2. Building and Installing Software
      1. Configuration of the Makefile
      2. Compilation
    3. Installation
    4. Building Software and Pipelines Wrap-up
  20. 15. Local Version Control
    1. What Is Version Control?
      1. The Lab Notebook of Computational Physics
      2. Version Control Tool Types
    2. Getting Started with Git
      1. Installing Git
      2. Getting Help (git --help)
      3. Control the Behavior of Git (git config)
    3. Local Version Control with Git
      1. Creating a Local Repository (git init)
      2. Staging Files (git add)
      3. Checking the Status of Your Local Copy (git status)
      4. Saving a Snapshot (git commit)
      5. git log: Viewing the History
      6. Viewing the Differences (git diff)
      7. Unstaging or Reverting a File (git reset)
      8. Discard Revisions (git revert)
      9. Listing, Creating, and Deleting Branches (git branch)
      10. Switching Between Branches (git checkout)
      11. Merging Branches (git merge)
      12. Dealing with Conflicts
    4. Version Conrol Wrap-Up
  21. 16. Remote Version Control
    1. Repository Hosting (github.com)
    2. Creating a Repository on GitHub
    3. Declaring a Remote (git remote)
    4. Sending Commits to Remote Repositories (git push)
    5. Downloading a Repository (git clone)
    6. Fetching the Contents of a Remote (git fetch)
    7. Merging the Contents of a Remote (git merge)
    8. Pull = Fetch and Merge (git pull)
    9. Conflicts
    10. Resolving Conflicts
    11. Remote Version Control Wrap-up
  22. 17. Debugging
    1. Encountering a Bug
    2. Print Statements
    3. Interactive Debugging
    4. Debugging in Python (pdb)
      1. Setting the Trace
      2. Stepping Forward
      3. Querying Variables
      4. Setting the State
      5. Running Functions and Methods
      6. Continuing the Execution
      7. Breakpoints
    5. Profiling
      1. Viewing the Profile with pstats
      2. Viewing the Profile Graphically
      3. Line Profiling with Kernprof
    6. Linting
    7. Debugging Wrap-up
  23. 18. Testing
    1. Why Do We Test?
    2. When Should We Test?
    3. Where Should We Write Tests?
    4. What and How to Test?
    5. Running Tests
    6. Edge Cases
      1. Corner Cases
    7. Unit Tests
    8. Integration Tests
    9. Regression Tests
    10. Test Generators
    11. Test Coverage
    12. Test-Driven Development
    13. Testing Wrap-up
  24. IV. Getting It Out There
  25. 19. Documentation
    1. Why Prioritize Documentation?
      1. Documentation Is Very Valuable
      2. Documentation Is Easier Than You Think
    2. Types of Documentation
      1. Theory Manuals
      2. User and Developer Guides
      3. Readme Files
      4. Comments
      5. Self-Documenting Code
      6. Docstrings
    3. Automation
      1. Sphinx
    4. Documentation Wrap-up
  26. 20. Publication
    1. Document Processing
      1. Separation of Content from Formatting
      2. Tracking Changes
    2. Text Editors
    3. Markup Languages
      1. LaTeX
      2. Bibliographies
    4. Publication Wrap-up
  27. 21. Collaboration
    1. Ticketing Systems
      1. Workflow Overview
      2. Creating an Issue
      3. Assigning an Issue
      4. Discussing an Issue
      5. Closing an Issue
    2. Pull Requests and Code Reviews
      1. Submitting a Pull Request
      2. Reviewing a Pull Request
      3. Merging a Pull Request
    3. Collaboration Wrap-up
  28. 22. Licenses, Ownership, and Copyright
    1. What Is Copyrightable?
    2. Right of First Publication
    3. What Is the Public Domain?
    4. Choosing a Software License
    5. Berkeley Software Distribution (BSD) License
    6. GNU General Public License (GPL)
    7. Creative Commons (CC)
    8. Other Licenses
    9. Changing the License
    10. Copyright Is Not Everything
    11. Licensing Wrap-up
  29. 23. Further Musings on Computational Physics
    1. Where to Go from Here
  30. Glossary
  31. Bibliography
  32. Index

Product information

  • Title: Effective Computation in Physics
  • Author(s): Anthony Scopatz, Kathryn D. Huff
  • Release date: June 2015
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491901533