May 2022
Beginner to intermediate
756 pages
21h 30m
Korean
book
HTTP 완벽 가이드
by David Gourlry, Brian Totty, Lee Eung Jun, Sang -Il Jeong, Anshu Agarwal, Seilu Reddy, Marjorie Sayre
Content preview from HTTP 완벽 가이드
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
248
HTTP
2
부 HTTP 아키텍처
9.1
크롤러와 크롤링
웹 크롤러는, 먼저 웹페이지를 한 개 가져오고, 그 다음 그 페이지가 가리키는 모든
웹페이지를 가져오고, 다시 그 페이지들이 가리키는 모든 웹페이지들을 가져오는,
이러한 일을 재귀적으로 반복하는 방식으로 웹을 순회하는 로봇이다. 웹 링크를 재
귀적으로 따라가는 로봇을 크롤러 혹은 스파이더라고 부르는데, HTML 하이퍼링크
들로 만들어진 웹을 따라 ‘기어다니기(crawl)’ 때문이다.
인터넷 검색엔진은 웹을 돌아다니면서 그들이 만나는 모든 문서를 끌어오기 위
해 크롤러를 사용한다. 이 문서들은 나중에 처리되어 검색 가능한 데이터베이스로
만들어지는데, 이는 사용자들이 특정 단어를 포함한 문서를 찾을 수 있게 해준다.
찾아서 가져와야 하는 페이지들이 수십억 개나 되다 보니 필연적으로 이들 검색엔
진 스파이더들은 가장 복잡한 로봇들 중 하나가 되었다. 크롤러들이 어떻게 동작하
는지 더 자세히 살펴보자.
9.1.1
어디에서 시작하는가: ‘루트 집합’
굶주린 크롤러를 풀어놓기 전에, 우선 출발지점을 주어야 한다. 크롤러가 방문을
시작하는 URL들의 초기 집합은 루트 집합(root set)이라고 불린다. 루트 집합을 고
를 때, 모든 링크를 크롤링하면 결과적으로 관심 있는 웹페이지들의 대부분을 가져
오게 될 수 있도록 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.