August 2017
Intermediate to advanced
500 pages
12h 3m
Japanese
Content preview from PythonとJavaScriptではじめるデータビジュアライゼーション
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
135
6
章
Scrapy
を使った
重量スクレイピング
スクレイピングの対象が大規模になると、
BeautifulSoup
と
requests
ライブラリを使ったスク
レイピングでは限界があります。リクエストはさらなるリクエストを生み出すので、スクレイピン
グしたデータの管理が困難になり、リクエストを同期的に行うと急速に処理に時間がかかるように
なります。おそらく予期しない問題が起こるでしょう。その時は、このような問題をすべて解決し
てくれる強力で堅牢なライブラリに頼る必要があります。そこで
Scrapy
の出番です。
BeautifulSoup
は簡単なスクレイピングに便利ですが、
Scrapy
は大規模なデータスクレイピン
グを容易に行うことができます。
Scrapy
は組み込みのキャッシュ（有効期限付き）、
Python
の
Twisted Web
フレームワークを介した非同期リクエスト、ユーザエージェントのランダム化など
の多くの期待する機能をすべて備えていますが、使いこなすのが難しいため、本章では簡単な例を
使って少しでも敷居を低くしています。
Scrapy
はとても強力で、データ可視化ツールの
1
つとし
て欠かせないものです。
Web
データ収集の可能性を確実に広げてくれますが、現時点で重量スク
レイピングが必要なければ、ノーベル賞データを収集したとみなして第Ⅲ部に進んでもかまいませ
ん。重量スクレイピングが必要なら、気を引き締めて本物のスクレイピングエンジンが実行できる
ことを確認してみましょう。
「
5.5
データのスクレイピング」 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.