August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
76
｜
第
3
章
URL
生成器
编写
URL
生成器是略微复杂的解决方案。对于按照层次结构组织的内容，例如
论坛、博客等来说，这种方式最为合适。
搜索引擎
通过搜索引擎查询特定的
URL
，并且只下载这些特定的
URL
。
站点地图
站点地图是一个名为
sitemap.xml
的文件（
https://oreil.ly/XANO0
），这是一种最
初为搜索引擎准备的标准。
sitemap.xml
文件包含网站上所有页面的列表（或指
向子站点地图的引用）。与
robots.txt
相反，这个文件名并不是固定的，有时由
robots.txt
指定。但一般我们都可以从网站的根目录找到
sitemap.xml
。
RSS
RSS
格式（
https://oreil.ly/_aOOM
）最初是为新闻源准备的，现在仍广泛用于订
阅频繁变化的内容源。
RSS
是一个
XML
文件，它不仅包含
URL
，而且还包含
文档标题，有时还包括文章摘要。
专用程序
你可以通过
GitHub
上的专用程序下载社交网络的数据以及类似的内容，例如利
用
Facebook Chat Downloader
（
https://oreil.ly/ThyNf
）下 载
Facebook
的聊天记录、
利用
Instaloader
（
https://oreil.ly/utGsC
）下载
Instagram
等。
在以下各节中，我们将重点介绍
robots.txt
、
sitemaps.xml
和
RSS
源。在本章的后面，
我们还将展示使用
URL
生成器分阶段下载的方法。
注意：请尽可能使用
API
下载数据
与生成
URL
，然后再下载并提取内容相比，使用 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.