At this point in our MapReduce run, the mappers have run and eventually write their intermediate output data to S3. Mappers were triggered by an invocation of SNS events on a given SNS topic. We will set up the reducers to be triggered based on an s3:ObjectCreated event. Taking a look at the serverless.yml file, we can see how I've done this:
functions: Reducer: handler: handler.reducer events: - s3: bucket: brianz-${env:ENV}-mapreduce-results event: s3:ObjectCreated:* rules: - suffix: '-done.csv'
The s3 section in the events block says: Whenever a new object is uploaded to s3 with the -done.csv suffix, invoke the hander.reducer function.
Just as the mapper was reasonably straightforward, so too is the reducer. Much ...
