Traversing multiple pages (Intermediate)

Up to this point, everything we have worked with has been scraping content from a single page, but where scraping really provides the most valuable results is in crawling and scraping multiple pages.

In the following recipe we will access the paginated result of a search query, visit each page of search results, and scrape all of the results.

How to do it...

  1. Enter the following code into a new PHP project:
    <?php // Function for making a GET request using cURL function curlGet($url) { $ch = curl_init(); // Initialising cURL session // Setting cURL options curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Returning transfer as a string curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow location curl_setopt($ch, ...

Get Instant PHP Web Scraping now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.