How to Deal with Flaky Tests

Flaky tests are like kryptonite to the automated tester—they sap your strength, slow you down, and generally just waste your time.

What’s a flaky test? Any test that doesn’t run reliably.

Say, for example, that you have a UI test that logs someone into the system, plays a song, and then logs them back out. When you run that test, it passes 99 times out of 100. That’s a flaky test because every once in awhile, when it does periodically fail, you have to stop what you are doing, run the test again, and then hope that it passes. Complete waste of time.

images/climb/flake.png

We want our automated tests (all of them) to be highly deterministic. ...

Get The Way of the Web Tester now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.