Data Science on AWS

Chapter 6. Prepare the Dataset for Model Training

In the previous chapter, we explored our dataset using SageMaker Studio and various Python-based visualization libraries. We gained some key business insights into our product catalog using the Amazon Customer Reviews Dataset. In addition, we analyzed summary statistics and performed quality checks on our dataset using SageMaker Processing Jobs, Apache Spark, and the AWS Deequ open source library.

In this chapter, we discuss how to transform human-readable text into machine-readable vectors in a process called “feature engineering.” Specifically, we will convert the raw review_body column from the Amazon Customer Reviews Dataset into BERT vectors. We use these BERT vectors to train and optimize a review-classifier model in Chapters 7 and 8, respectively. We will also dive deep into the origins of natural language processing and BERT in Chapter 7.

We will use the review-classifier model to predict the star_rating of product reviews from social channels, partner websites, etc. By predicting the star_rating of reviews in the wild, the product management and customer service teams can use these predictions to address quality issues as they escalate publicly—not wait for a direct inbound email or phone call. This reduces the mean time to detect quality issues down to minutes/hours from days/months.

Perform Feature Selection and Engineering

AI and machine learning algorithms are numerical-optimization methods that operate on numbers ...