book

Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference

Name: Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference
Author: Cameron Davidson-Pilon
ISBN: 9780133902914

by Cameron Davidson-Pilon

October 2015

Beginner to intermediate

300 pages

7h 19m

English

Addison-Wesley Professional

Read now

Unlock full access

About This eBook
Title Page
Copyright Page
Dedication Page
Contents
Foreword
Preface
Acknowledgments
About the Author
1. The Philosophy of Bayesian Inference
1.1 Introduction1.1.1 The Bayesian State of Mind1.1.2 Bayesian Inference in Practice1.1.3 Are Frequentist Methods Incorrect?1.1.4 A Note on “Big Data”1.2 Our Bayesian Framework1.2.1 Example: Mandatory Coin-Flip1.2.2 Example: Librarian or Farmer?1.3 Probability Distributions1.3.1 Discrete Case1.3.2 Continuous Case1.3.3 But What Is λ?1.4 Using Computers to Perform Bayesian Inference for Us1.4.1 Example: Inferring Behavior from Text-Message Data1.4.2 Introducing Our First Hammer: PyMC1.4.3 Interpretation1.4.4 What Good Are Samples from the Posterior, Anyway?1.5 Conclusion1.6 Appendix1.6.1 Determining Statistically if the Two λs Are Indeed Different?1.6.2 Extending to Two Switchpoints1.7 Exercises1.7.1 Answers1.8 References

2. A Little More on PyMC
2.1 Introduction2.1.1 Parent and Child Relationships2.1.2 PyMC Variables2.1.3 Including Observations in the Model2.1.4 Finally. . .2.2 Modeling Approaches2.2.1 Same Story, Different Ending2.2.2 Example: Bayesian A/B Testing2.2.3 A Simple Case2.2.4 A and B Together2.2.5 Example: An Algorithm for Human Deceit2.2.6 The Binomial Distribution2.2.7 Example: Cheating Among Students2.2.8 Alternative PyMC Model2.2.9 More PyMC Tricks2.2.10 Example: Challenger Space Shuttle Disaster2.2.11 The Normal Distribution2.2.12 What Happened the Day of the Challenger Disaster?2.3 Is Our Model Appropriate?2.3.1 Separation Plots2.4 Conclusion2.5 Appendix2.6 Exercises2.6.1 Answers2.7 References
3. Opening the Black Box of MCMC
3.1 The Bayesian Landscape3.1.1 Exploring the Landscape Using MCMC3.1.2 Algorithms to Perform MCMC3.1.3 Other Approximation Solutions to the Posterior3.1.4 Example: Unsupervised Clustering Using a Mixture Model3.1.5 Don’t Mix Posterior Samples3.1.6 Using MAP to Improve Convergence3.2 Diagnosing Convergence3.2.1 Autocorrelation3.2.2 Thinning3.2.3 pymc.Matplot.plot()3.3 Useful Tips for MCMC3.3.1 Intelligent Starting Values3.3.2 Priors3.3.3 The Folk Theorem of Statistical Computing3.4 Conclusion3.5 References
4. The Greatest Theorem Never Told
4.1 Introduction4.2 The Law of Large Numbers4.2.1 Intuition4.2.2 Example: Convergence of Poisson Random Variables4.2.3 How Do We Compute Var(Z)?4.2.4 Expected Values and Probabilities4.2.5 What Does All This Have to Do with Bayesian Statistics?4.3 The Disorder of Small Numbers4.3.1 Example: Aggregated Geographic Data4.3.2 Example: Kaggle’s U.S. Census Return Rate Challenge4.3.3 Example: How to Sort Reddit Comments4.3.4 Sorting!4.3.5 But This Is Too Slow for Real-Time!4.3.6 Extension to Starred Rating Systems4.4 Conclusion4.5 Appendix4.5.1 Derivation of Sorting Comments Formula4.6 Exercises4.6.1 Answers4.7 References
5. Would You Rather Lose an Arm or a Leg?
5.1 Introduction5.2 Loss Functions5.2.1 Loss Functions in the Real World5.2.2 Example: Optimizing for the Showcase on The Price Is Right5.3 Machine Learning via Bayesian Methods5.3.1 Example: Financial Prediction5.3.2 Example: Kaggle Contest on Observing Dark Worlds5.3.3 The Data5.3.4 Priors5.3.5 Training and PyMC Implementation5.4 Conclusion5.5 References
6. Getting Our Priorities Straight
6.1 Introduction6.2 Subjective versus Objective Priors6.2.1 Objective Priors6.2.2 Subjective Priors6.2.3 Decisions, Decisions . . .6.2.4 Empirical Bayes6.3 Useful Priors to Know About6.3.1 The Gamma Distribution6.3.2 The Wishart Distribution6.3.3 The Beta Distribution6.4 Example: Bayesian Multi-Armed Bandits6.4.1 Applications6.4.2 A Proposed Solution6.4.3 A Measure of Good6.4.4 Extending the Algorithm6.5 Eliciting Prior Distributions from Domain Experts6.5.1 Trial Roulette Method6.5.2 Example: Stock Returns6.5.3 Pro Tips for the Wishart Distribution6.6 Conjugate Priors6.7 Jeffreys Priors6.8 Effect of the Prior as N Increases6.9 Conclusion6.10 Appendix6.10.1 Bayesian Perspective of Penalized Linear Regressions6.10.2 Picking a Degenerate Prior6.11 References
7. Bayesian A/B Testing
7.1 Introduction7.2 Conversion Testing Recap7.3 Adding a Linear Loss Function7.3.1 Expected Revenue Analysis7.3.2 Extending to an A/B Experiment7.4 Going Beyond Conversions: t-test7.4.1 The Setup of the t-test7.5 Estimating the Increase7.5.1 Creating Point Estimates7.6 Conclusion7.7 References
Glossary
Index
Code Snippets

Content preview from Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference

Preface

The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chapters of slow, mathematical analysis. The typical text on Bayesian inference involves two to three chapters on probability theory, then enters into what Bayesian inference is. Unfortunately, due to the mathematical intractability of most Bayesian models, the reader is only shown simple, artificial examples. This can leave the user with a “So what?” feeling about Bayesian inference. In fact, this was my own prior opinion.

After some recent success of Bayesian methods in machine-learning competitions, I decided to investigate the subject again. Even with my mathematical background, it took me three straight days of reading examples and trying ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9780133902914Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference

by Cameron Davidson-Pilon

Preface

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.