book

Effective Computation in Physics

by Anthony Scopatz, Kathryn D. Huff

June 2015

Intermediate to advanced

550 pages

13h 27m

English

O'Reilly Media, Inc.

Read now

Unlock full access

What Is This Book?Who This Book Is ForWho This Book Is Not ForCase Study on How to Use This Book: Radioactive Decay ConstantsAccessing Data and LibrariesCreating a Simple ProgramAutomating Data CollectionAnalyzing and Plotting the DataKeeping Track of ChangesTesting the CodeDocumenting the CodePublishingWhat to Do While Reading This BookConventions Used in This BookUsing Code ExamplesInstallation and SetupStep 1: Download and Install Miniconda (or Anaconda)Step 2: Install the PackagesSafari® Books OnlineHow to Contact UsAcknowledgments
Navigating the ShellThe Shell Is a Programming LanguagePaths and pwdHome Directory (~)Listing the Contents (ls)Changing Directories (cd)File Inspection (head and tail)Manipulating Files and DirectoriesCreating Files (nano, emacs, vi, cat, >, and touch)Copying and Renaming Files (cp and mv)Making Directories (mkdir)Deleting Files and Directories (rm)Flags and WildcardsGetting HelpReading the Manual (man)Finding the Right Hammer (apropos)Combining Utilities with Redirection and Pipes (>, >>, and |)Permissions and SharingSeeing Permissions (ls -l)Setting Ownership (chown)Setting Permissions (chmod)Creating Links (ln)Connecting to Other Computers (ssh and scp)The EnvironmentSaving Environment Variables (.bashrc)Running Programs (PATH)Nicknaming Commands (alias)Scripting with BashCommand Line Wrap-up
Running PythonCommentsVariablesSpecial VariablesBoolean ValuesNone Is Not Zero!NotImplemented Is Not None!OperatorsStringsString IndexingString ConcatenationString LiteralsString MethodsModulesImporting ModulesImporting Variables from a ModuleAliasing ImportsAliasing Variables on ImportPackagesThe Standard Library and the Python EcosystemPython Wrap-up
ListsTuplesSetsDictionariesContainers Wrap-up
Conditionalsif-else Statementsif-elif-else Statementsif-else ExpressionExceptionsRaising ExceptionsLoopswhile Loopsfor LoopsComprehensionsFlow Control and Logic Wrap-up
Functions in PythonKeyword ArgumentsVariable Number of ArgumentsMultiple Return ValuesScopeRecursionLambdasGeneratorsDecoratorsFunction Wrap-up
Object OrientationObjectsClassesClass VariablesInstance VariablesConstructorsMethodsStatic MethodsDuck TypingPolymorphismDecorators and MetaclassesObject Orientation Wrap-up

Preparing DataExperimental DataSimulation DataMetadataLoading DataNumPyPyTablesPandasBlazeCleaning and Munging DataMissing DataAnalysisModel-Driven AnalysisData-Driven AnalysisVisualizationVisualization ToolsGnuplotmatplotlibBokehInkscapeAnalysis and Visualization Wrap-up
Messy MagnetismMetacharacters on the Command LineListing Files with Simple PatternsGlobally Finding Filenames with Patterns (find)grep, sed, and awkFinding Patterns in Files (grep)Finding and Replacing Patterns in Files (sed)Finding and Replacing a Complex Patternsed ExtrasManipulating Columns of Data (awk)Python Regular ExpressionsRegular Expressions Wrap-up
ArraysdtypesSlicing and ViewsArithmetic and BroadcastingFancy IndexingMaskingStructured ArraysUniversal FunctionsOther Valuable FunctionsNumPy Wrap-up
Files in PythonAn Aside About Computer ArchitectureBig Ideas in HDF5File ManipulationsHierarchy LayoutChunkingIn-Core and Out-of-Core OperationsIn-CoreOut-of-CoreQueryingCompressionHDF5 UtilitiesStoring Data Wrap-up
Hash TablesResizingCollisionsData FramesSeriesThe Data Frame StructureB-TreesK-D TreesData Structures Wrap-up
Scale and ScalabilityProblem ClassificationExample: N-Body ProblemNo ParallelismThreadsMultiprocessingMPIParallelism Wrap-up
Deploying the Software ItselfpipCondaVirtual MachinesDockerDeploying to the CloudDeploying to SupercomputersDeployment Wrap-up
makeRunning makeMakefilesTargetsSpecial TargetsBuilding and Installing SoftwareConfiguration of the MakefileCompilationInstallationBuilding Software and Pipelines Wrap-up
What Is Version Control?The Lab Notebook of Computational PhysicsVersion Control Tool TypesGetting Started with GitInstalling GitGetting Help (git --help)Control the Behavior of Git (git config)Local Version Control with GitCreating a Local Repository (git init)Staging Files (git add)Checking the Status of Your Local Copy (git status)Saving a Snapshot (git commit)git log: Viewing the HistoryViewing the Differences (git diff)Unstaging or Reverting a File (git reset)Discard Revisions (git revert)Listing, Creating, and Deleting Branches (git branch)Switching Between Branches (git checkout)Merging Branches (git merge)Dealing with ConflictsVersion Conrol Wrap-Up
Repository Hosting (github.com)Creating a Repository on GitHubDeclaring a Remote (git remote)Sending Commits to Remote Repositories (git push)Downloading a Repository (git clone)Fetching the Contents of a Remote (git fetch)Merging the Contents of a Remote (git merge)Pull = Fetch and Merge (git pull)ConflictsResolving ConflictsRemote Version Control Wrap-up
Encountering a BugPrint StatementsInteractive DebuggingDebugging in Python (pdb)Setting the TraceStepping ForwardQuerying VariablesSetting the StateRunning Functions and MethodsContinuing the ExecutionBreakpointsProfilingViewing the Profile with pstatsViewing the Profile GraphicallyLine Profiling with KernprofLintingDebugging Wrap-up
Why Do We Test?When Should We Test?Where Should We Write Tests?What and How to Test?Running TestsEdge CasesCorner CasesUnit TestsIntegration TestsRegression TestsTest GeneratorsTest CoverageTest-Driven DevelopmentTesting Wrap-up
Why Prioritize Documentation?Documentation Is Very ValuableDocumentation Is Easier Than You ThinkTypes of DocumentationTheory ManualsUser and Developer GuidesReadme FilesCommentsSelf-Documenting CodeDocstringsAutomationSphinxDocumentation Wrap-up
Document ProcessingSeparation of Content from FormattingTracking ChangesText EditorsMarkup LanguagesLaTeXBibliographiesPublication Wrap-up
Ticketing SystemsWorkflow OverviewCreating an IssueAssigning an IssueDiscussing an IssueClosing an IssuePull Requests and Code ReviewsSubmitting a Pull RequestReviewing a Pull RequestMerging a Pull RequestCollaboration Wrap-up
What Is Copyrightable?Right of First PublicationWhat Is the Public Domain?Choosing a Software LicenseBerkeley Software Distribution (BSD) LicenseGNU General Public License (GPL)Creative Commons (CC)Other LicensesChanging the LicenseCopyright Is Not EverythingLicensing Wrap-up
Where to Go from Here

Content preview from Effective Computation in Physics

Chapter 2. Programming Blastoff with Python

A lot of people talk about the greatness of Python as a programming language. They are right! But what is Python, and why is it so awesome? Python is a general-purpose, dynamic, high-level language that is easy to learn. Python is also known as a glue language because it plays nicely with other languages, including C, C++, and Fortran. For these reasons it has established a strong foothold as a data analysis language. This makes it popular in science and engineering, and in physics-related fields in particular.

The main criticism of Python is its speed. Python is an interpreted language, which makes it more similar to R, Ruby, and MATLAB than it is to compiled languages like C, C++, or Fortran. Pythonistas everywhere live by the mantra that “premature optimization is bad.” Concerns that it is too slow are often refuted with these arguments:

Developer time—the time the programmer spends programming—is more valuable than execution time.¹
Most speed issues can be overcome by using the appropriate data structures and algorithms.
If you really have a need for speed you can always write the performance-critical parts in a compiled language and then expose the functionality to Python.

The most important aspect of Python is it is fun to use! The more you learn, the more you want to learn, and the more you find there is to learn. The Python ecosystem is extraordinarily rich and the community members are, by and large, friendly. Unfortunately, ...