How to Deal with Flaky Tests

Flaky tests are like kryptonite to the automated tester—they sap your strength, slow you down, and generally just waste your time.

What’s a flaky test? Any test that doesn’t run reliably.

Say, for example, that you have a UI test that logs someone into the system, plays a song, and then logs them back out. When you run that test, it passes 99 times out of 100. That’s a flaky test because every once in awhile, when it does periodically fail, you have to stop what you are doing, run the test again, and then hope that it passes. Complete waste of time.

images/climb/flake.png

We want our automated tests (all of them) to be highly deterministic. ...

Get The Way of the Web Tester now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.