October 2021
Beginner to intermediate
416 pages
9h 3m
Korean
Content preview from 살아 움직이는 머신러닝 파이프라인 설계: 텐서플로와 함께하는 머신러닝 프로젝트 자동화
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
58
살아 움직이는 머신러닝 파이프라인 설계
2.7.2
기본 데이터 파이프라인
아파치 빔의 추상화는 컬렉션과 변환이라는 두 가지 개념을 기반으로 합니다. 아파치 빔의 컬
렉션은 지정된 파일 또는 스트림에서 데이터를 읽거나 쓰는 작업을 설명합니다. 반면에 아파치
빔의 변환은 데이터를 조작하는 방법을 설명합니다. 모든 컬렉션과 변환은 파이프라인의 콘텍
스트에서 실행됩니다(콘텍스트 매니저
context
manager
명령을 통해 파이썬으로 표시됨 ). 다음 예
제에서 컬렉션 또는 변환을 정의할 때 실제로 로드되거나 변환되는 데이터는 없습니다. 이 문
제는 파이프라인이 런타임 환경 (예: 아파치 빔의
DirectRunner
, 아파치 스파크, 아파치 플링
크
Apache
Flink
, 구글 클라우드 데이터플로)의 콘텍스트에서 실행될 때에만 발생합니다.
기본 컬렉션 예제
데이터 파이프라인은 대개 데이터를 읽거나 쓰며 시작하고 끝납니다. 데이터 파이프라인은 보
통
PCollections
라는 컬렉션을 통해 아파치 빔에서 처리됩니다. 그런 다음 컬렉션을 변환하
고 최종 결과는 다시 컬렉션으로 표현되어 파일 시스템에 기록할 수 있습니다.
다음 예는 텍스트 파일을 읽고 모든 행을 반환하는 방법입니다.
import apache_beam as beam
with beam.Pipeline() as p: ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.