Summary
Having read this chapter, you should now understand the fundamentals of web scraping, such as performing an HTTP GET request and searching for a string using string matching or regular expressions to find HTML comments, emails, and other keywords. You should also understand how to extract the HTTP headers and set custom headers to set cookies and custom user agent strings. Moreover, you should understand the basic concepts of fingerprinting and have some idea of how to gather information about a web application based on the source code provided.
Having worked through this chapter, you should also understand the basics of using the goquery package to find HTML elements in the DOM in a jQuery style. You should feel comfortable finding ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access