book

Effective Computation in Physics

by Anthony Scopatz, Kathryn D. Huff

June 2015

Intermediate to advanced

550 pages

13h 27m

English

O'Reilly Media, Inc.

Read now

Unlock full access

What Is This Book?Who This Book Is ForWho This Book Is Not ForCase Study on How to Use This Book: Radioactive Decay ConstantsAccessing Data and LibrariesCreating a Simple ProgramAutomating Data CollectionAnalyzing and Plotting the DataKeeping Track of ChangesTesting the CodeDocumenting the CodePublishingWhat to Do While Reading This BookConventions Used in This BookUsing Code ExamplesInstallation and SetupStep 1: Download and Install Miniconda (or Anaconda)Step 2: Install the PackagesSafari® Books OnlineHow to Contact UsAcknowledgments
Navigating the ShellThe Shell Is a Programming LanguagePaths and pwdHome Directory (~)Listing the Contents (ls)Changing Directories (cd)File Inspection (head and tail)Manipulating Files and DirectoriesCreating Files (nano, emacs, vi, cat, >, and touch)Copying and Renaming Files (cp and mv)Making Directories (mkdir)Deleting Files and Directories (rm)Flags and WildcardsGetting HelpReading the Manual (man)Finding the Right Hammer (apropos)Combining Utilities with Redirection and Pipes (>, >>, and |)Permissions and SharingSeeing Permissions (ls -l)Setting Ownership (chown)Setting Permissions (chmod)Creating Links (ln)Connecting to Other Computers (ssh and scp)The EnvironmentSaving Environment Variables (.bashrc)Running Programs (PATH)Nicknaming Commands (alias)Scripting with BashCommand Line Wrap-up
Running PythonCommentsVariablesSpecial VariablesBoolean ValuesNone Is Not Zero!NotImplemented Is Not None!OperatorsStringsString IndexingString ConcatenationString LiteralsString MethodsModulesImporting ModulesImporting Variables from a ModuleAliasing ImportsAliasing Variables on ImportPackagesThe Standard Library and the Python EcosystemPython Wrap-up
ListsTuplesSetsDictionariesContainers Wrap-up
Conditionalsif-else Statementsif-elif-else Statementsif-else ExpressionExceptionsRaising ExceptionsLoopswhile Loopsfor LoopsComprehensionsFlow Control and Logic Wrap-up
Functions in PythonKeyword ArgumentsVariable Number of ArgumentsMultiple Return ValuesScopeRecursionLambdasGeneratorsDecoratorsFunction Wrap-up
Object OrientationObjectsClassesClass VariablesInstance VariablesConstructorsMethodsStatic MethodsDuck TypingPolymorphismDecorators and MetaclassesObject Orientation Wrap-up

Preparing DataExperimental DataSimulation DataMetadataLoading DataNumPyPyTablesPandasBlazeCleaning and Munging DataMissing DataAnalysisModel-Driven AnalysisData-Driven AnalysisVisualizationVisualization ToolsGnuplotmatplotlibBokehInkscapeAnalysis and Visualization Wrap-up
Messy MagnetismMetacharacters on the Command LineListing Files with Simple PatternsGlobally Finding Filenames with Patterns (find)grep, sed, and awkFinding Patterns in Files (grep)Finding and Replacing Patterns in Files (sed)Finding and Replacing a Complex Patternsed ExtrasManipulating Columns of Data (awk)Python Regular ExpressionsRegular Expressions Wrap-up
ArraysdtypesSlicing and ViewsArithmetic and BroadcastingFancy IndexingMaskingStructured ArraysUniversal FunctionsOther Valuable FunctionsNumPy Wrap-up
Files in PythonAn Aside About Computer ArchitectureBig Ideas in HDF5File ManipulationsHierarchy LayoutChunkingIn-Core and Out-of-Core OperationsIn-CoreOut-of-CoreQueryingCompressionHDF5 UtilitiesStoring Data Wrap-up
Hash TablesResizingCollisionsData FramesSeriesThe Data Frame StructureB-TreesK-D TreesData Structures Wrap-up
Scale and ScalabilityProblem ClassificationExample: N-Body ProblemNo ParallelismThreadsMultiprocessingMPIParallelism Wrap-up
Deploying the Software ItselfpipCondaVirtual MachinesDockerDeploying to the CloudDeploying to SupercomputersDeployment Wrap-up
makeRunning makeMakefilesTargetsSpecial TargetsBuilding and Installing SoftwareConfiguration of the MakefileCompilationInstallationBuilding Software and Pipelines Wrap-up
What Is Version Control?The Lab Notebook of Computational PhysicsVersion Control Tool TypesGetting Started with GitInstalling GitGetting Help (git --help)Control the Behavior of Git (git config)Local Version Control with GitCreating a Local Repository (git init)Staging Files (git add)Checking the Status of Your Local Copy (git status)Saving a Snapshot (git commit)git log: Viewing the HistoryViewing the Differences (git diff)Unstaging or Reverting a File (git reset)Discard Revisions (git revert)Listing, Creating, and Deleting Branches (git branch)Switching Between Branches (git checkout)Merging Branches (git merge)Dealing with ConflictsVersion Conrol Wrap-Up
Repository Hosting (github.com)Creating a Repository on GitHubDeclaring a Remote (git remote)Sending Commits to Remote Repositories (git push)Downloading a Repository (git clone)Fetching the Contents of a Remote (git fetch)Merging the Contents of a Remote (git merge)Pull = Fetch and Merge (git pull)ConflictsResolving ConflictsRemote Version Control Wrap-up
Encountering a BugPrint StatementsInteractive DebuggingDebugging in Python (pdb)Setting the TraceStepping ForwardQuerying VariablesSetting the StateRunning Functions and MethodsContinuing the ExecutionBreakpointsProfilingViewing the Profile with pstatsViewing the Profile GraphicallyLine Profiling with KernprofLintingDebugging Wrap-up
Why Do We Test?When Should We Test?Where Should We Write Tests?What and How to Test?Running TestsEdge CasesCorner CasesUnit TestsIntegration TestsRegression TestsTest GeneratorsTest CoverageTest-Driven DevelopmentTesting Wrap-up
Why Prioritize Documentation?Documentation Is Very ValuableDocumentation Is Easier Than You ThinkTypes of DocumentationTheory ManualsUser and Developer GuidesReadme FilesCommentsSelf-Documenting CodeDocstringsAutomationSphinxDocumentation Wrap-up
Document ProcessingSeparation of Content from FormattingTracking ChangesText EditorsMarkup LanguagesLaTeXBibliographiesPublication Wrap-up
Ticketing SystemsWorkflow OverviewCreating an IssueAssigning an IssueDiscussing an IssueClosing an IssuePull Requests and Code ReviewsSubmitting a Pull RequestReviewing a Pull RequestMerging a Pull RequestCollaboration Wrap-up
What Is Copyrightable?Right of First PublicationWhat Is the Public Domain?Choosing a Software LicenseBerkeley Software Distribution (BSD) LicenseGNU General Public License (GPL)Creative Commons (CC)Other LicensesChanging the LicenseCopyright Is Not EverythingLicensing Wrap-up
Where to Go from Here

Content preview from Effective Computation in Physics

Bibliography

Albrecht, Michael, Patrick Donnelly, Peter Bui, and Douglas Thain. “Makeflow: A Portable Abstraction for Data Intensive Computing on Clusters, Clouds, and Grids.” Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies (2012): 1–13. ACM Press. doi:10.1145/2443416.2443417.

Alted, F. “Why Modern CPUs Are Starving and What Can Be Done about It.” Computing in Science & Engineering 12 (2010): 68–71. doi:10.1109/MCSE.2010.51.

Barendregt, Henk P., and Erik Barendsen. “Introduction to Lambda Calculus.” Nieuw archief voor wisenkunde 4 (1984): 337–372.

Beck, Kent. Test-Driven Development: By Examples. Boston, MA: Addison-Wesley, 2002.

Chacon, Scott, and Ben Straub. Pro Git. 2nd ed. New York: Apress, 2014. http://git-scm.com/book/en/v2.

Collette, Andrew. Python and HDF5: Unlocking Scientific Data. Sebastopol, CA: O’Reilly Media, 2013.

Donoho, David L., Arian Maleki, Inam Ur Rahman, Morteza Shahram, and Victoria Stodden. “Reproducible Research in Computational Harmonic Analysis.” Computing in Science & Engineering (11): 8–18. doi:10.1109/MCSE.2009.15.

Drepper, Ulrich. “What Every Programmer Should Know About Memory.” September 21, 2007. http://lwn.net/Articles/250967/.

Duarte, Gustavo. “What Your Computer Does While You Wait.” November 30, 2008. http://bit.ly/comp-wait.

Feathers, Michael. Working Effectively with Legacy Code. Upper Saddle River, NJ: Prentice Hall, 2004.

Fenner, Martin. “One-Click ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Learn Physics with Functional Programming

Scott N. Walck

Math for Deep Learning

Ronald T. Kneusel

Effective Python

Brett Slatkin

Good Math

Mark C. Chu-Carroll

Publisher Resources

ISBN: 9781491901564Errata Page Supplemental Content

Bibliography

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Learn Physics with Functional Programming

Math for Deep Learning

Effective Python

Good Math

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.