PHP 7 Programming Cookbook

Building a deep web scanner

Sometimes you need to scan a website, but go one level deeper. For example, you want to build a web tree diagram of a website. This can be accomplished by looking for all <A> tags and following the HREF attributes to the next web page. Once you have acquired the child pages, you can then continue scanning in order to complete the tree.

How to do it...

1. A core component of a deep web scanner is a basic Hoover class, as described previously. The basic procedure presented in this recipe is to scan the target website and hoover up all the HREF attributes. For this purpose, we define a Application\Web\Deep class. We add a property that represents the DNS domain:

   namespace Application\Web;
   class Deep
   {
       protected $domain;

2. Next, ...