O'Reilly logo

Effective Amazon Machine Learning by Alexis Perrier

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Adding Redshift to the Kinesis Firehose

This part is delicate as several pieces from different services must fit together:

  • The data structure, table declaration, and Kinesis Redshift configuration
  • The data fields aggregation and subsequent parsing
  • The role and associated policies

The fields of the Redshift table that stores the data need to be synchronized in three different places:

  1. The Redshift table with a proper definition of fields.
  2. The script that sends the data to Kinesis. Depending on how the record sent to Kinesis is aggregated together and later parsed by Redshift, the script must concatenate the same number of fields in the same order as the ones defined in the Redshift table. For instance, when we write record = ','.join([str(st.id) ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required