May 2022
Beginner to intermediate
756 pages
21h 30m
Korean
book
HTTP 완벽 가이드
by David Gourlry, Brian Totty, Lee Eung Jun, Sang -Il Jeong, Anshu Agarwal, Seilu Reddy, Marjorie Sayre
Content preview from HTTP 완벽 가이드
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
258
HTTP
2
부 HTTP 아키텍처
URL
/사이트 블랙리스트
로봇 순환을 만들어 내거나 함정인 것으로 알려진 사이트와 URL의 목록을 만들어
관리하고 그들을 전염병 피하듯 피한다. 문제를 일으키는 사이트나 URL이 발견될
때마다 이 블랙리스트에 추가한다.
이는 사람의 손을 필요로 한다. 그러나 오늘날 실제 사용 중인 대부분의 대규모
크롤러들은 어떤 문제를 내재하고 있거나 악의적인 것이 확실한 사이트들을 피하
기 위해 사용하는 몇 가지 형태의 블랙리스트를 가지고 있다. 블랙리스트는 크롤링
되는 것을 싫어하는 특정 사이트를 피하기 위해 사용될 수 있다.
7
패턴 발견
파일 시스템의 심벌릭 링크를 통한 순환과 그와 비슷한 오설정들은 일정 패턴을 따
르는 경향이 있다. 예를 들어, URL은 중복된 구성 요소와 함께 점점 길어질 수도 있
다. 몇몇 로봇은 반복되는 구성요소를 가진 URL을 잠재적인 순환으로 보고, 둘 혹
은 셋 이상의 반복된 구성요소를 갖고 있는 URL을 크롤링하는 것을 거절한다.
반복이 ‘/subdir/subdir/subdir...’과 같이 단순한 것만 있는 것은 아니다. ‘/subdir/
images/subdir/images/subdir/images/...’과 같이 주기의 길이가 2 이상인 반복도
있을 수 있다. 몇몇 로봇은 몇 가지 다른 주기의 반복 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.