Summary
In this chapter, we have learned how to write a scraping script using the rvest library. First, we worked on how to collect URLs, and then we worked on how to create XPath rules. After that, we wrote our first web scraping script using the rvest library. We have applied basic text and manipulation techniques. Once we had the final data ready, we calculated the daily average comment counts, and at the final step, we created a line plot using basic R plots and plotly plots to visualize the average comment count, article counts for each categories, and the summary statistics of the comment counts.
In the next chapter, we will talk about the fundamentals of Selenium and how to do web scraping using the RSelenium library. At the end of ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access