How to Deal with Flaky Tests
Flaky tests are like kryptonite to the automated tester—they sap your strength, slow you down, and generally just waste your time.
What’s a flaky test? Any test that doesn’t run reliably.
Say, for example, that you have a UI test that logs someone into the system, plays a song, and then logs them back out. When you run that test, it passes 99 times out of 100. That’s a flaky test because every once in awhile, when it does periodically fail, you have to stop what you are doing, run the test again, and then hope that it passes. Complete waste of time.
We want our automated tests (all of them) to be highly deterministic. ...