November 2023
Beginner to intermediate
344 pages
3h 47m
Korean
Content preview from 데이터 엔지니어를 위한 97가지 조언
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
82 분산 프로그래밍에서 중요한 개념 3가지
아디 폴락(Adi Polak)
많은 데이터 엔지니어가 ETL이나 ELT 작업을 위한 파이프라인을 만듭니다. 변환(T) 작업 중에는 컴퓨터 하나의 메모리에 맞는 데이터로 작업할 수 있습니다. 그렇지만 데이터는 종종 목표하는 바를 이루기 위해 분산 병렬 계산을 이용하는 프레임워크나 솔루션을 이용해야 하는 경우가 많습니다. 이런 요구에 맞추기 위해 많은 연구자가 아파치 스파크, 아파치 카산드라, 아파치 카프카, 텐서플로 등과 같은 알려진 프레임워크에 구현된 분산 프로그래밍 및 계산 모델을 개발했습니다. 지금부터 데이터 분석 및 분산 머신 러닝에 널리 사용되는 분산 프로그래밍 모델 3가지를 살펴보겠습니다.
¦ 맵리듀스 알고리즘 ¦
맵리듀스는 구글이 2004년에 개발한 분산 계산 알고리즘입니다. 개발자는 map 함수와 reduce 함수를 지정해야 합니다. map 함수는 키/값 쌍을 처리해서 중간 결과로 여러 개의 키/값 쌍을 생성하며, reduce 함수는 중간 결과에서 동일한 키를 가진 모든 값을 병합합니다. 이 방식은 데이터 분석에서의 분리-적용-결합split-apply-combine 전략을 확장한 것입니다.
실제로 모든 작업은 여러 개의 map 함수와 reduce 함수로 나뉩니다. 데이터는 여러 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.