March 2024
Beginner to intermediate
416 pages
6h 54m
Japanese
Content preview from データエンジニアリングの基礎 ―データプロジェクトで失敗しないために
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
7章データ取り込み
これまでに、データエンジニアとして遭遇するであろうソースシステムと、データの保存方法について学んだ。次は、さまざまなソースシステムからデータを取り込む際に適用されるパターンと選択肢について考えてみよう。本章では、データの取り込み（data ingestion、図7-1を参照）について説明する。取り込みフェーズで考慮すべき重要なエンジニアリング上の課題、2つの主要なパターン（バッチ取り込みとストリーミング取り込み）、遭遇するであろう技術、データ取り込みパイプラインを開発する際に協力する相手、取り込みフェーズにおける底流について議論する。
図7-1 データ処理を行うには、まずデータを取り込む必要がある
7.1 データ取り込みとは
データ取り込みとは、データをある場所から別の場所に移動するプロセスである。データ取り込みは、データエンジニアリングライフサイクルにおいて、ソースシステムからストレージへデータを移動することを意味する。取り込みはこの2つの中間にあたる（図7-2）。
図7-2 ソースシステムからストレージにデータを取り込む
ここで、データ取り込みとデータ統合を簡単に比較しておこう。データ取り込みがA地点からB地点へのデータ移動であるのに対し、データ ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.