Julia 1.0 Programming Complete Reference Guide

Learning about HTTP headers

HTTP headers allow the client and the server to pass additional information. We won't go into the details of header transmission since Julia's HTTP library saves us from having to deal with raw headers. However, a few are worth mentioning, as they are important for web scraping:

• Age, Cache-Control, and Expires represent the validity of the page and can be used to set data refresh times
• Last-Modified, Etag, and If-Modified-Since can be used for content versioning, to check if the page has changed since it was last retrieved
• Cookie and Set-Cookie have to be used in order to read and write cookies that are required for correct communication with the server
• The Content-* family of headers, such as Content-Disposition ...