Managing and Mitigating Bots

Chapter 13. Identifying Automated Traffic

Any effective bot management strategy involves two separate steps: first, you must identify which traffic is non-human and within that definition whether it is non-human traffic that you need to take action against. After you have made that assessment you will then need to decide what action to take.

This chapter discusses the first, and by far the most complex, of those two steps.

Indications of an Automated Traffic Problem

A common question website owners will ask is “Are there any obvious symptoms I can look at to see if I have a bot problem?”

Keep in mind that automated traffic is often doing its best to make itself not noticed, so there may not be any obvious symptoms.

However, if you see any of the following things on your site it does suggest that a more thorough investigation of automated traffic could be worthwhile:

• High infrastructure usage that is not associated with known/expected increases in traffic

• Traffic patterns that are unusually spiky or do not follow the pattern of usage that you would expect

• Increases in visitor numbers or infrastructure usage that is not associated with an increase in conversions

• Increased numbers of failed payments

• Increased numbers of failed logins

• Increased numbers of chargebacks

• Increased levels of IVT reported by ad exchanges and SSP

Challenges

The challenge of identifying non-human traffic is simple: there is no difference between a request made by a human and that made by a non-human ...