Job Ready Go

Lesson 21Pulling It All Together: Word Analysis in Go

In this lesson, you will apply what you've learned up to this point to perform a common text analysis process using Go. Specifically, you will build a program that takes a dataset of e-commerce reviews and analyzes them to calculate the number of times each word appears.

LEARNING OBJECTIVES

By the end of this lesson, you will be able to build a project that can:

• Read a JSON file that contains a list of online e-commerce reviews.
• Tokenize each review in the dataset.
• Compute word count in each review in the dataset.

EXAMINING THE DATA

When you start any data analysis process, the first step is to ensure that the data is in a format that your system can use and ensure that the data is available for use. For our project, you will need to download the data. You'll use the Digital Music review set from Julian McAuley's Amazon product data website, which can be found at http://jmcauley.ucsd.edu/data/amazon. This file, reviews.json, can also be found in the downloadable zip file for this book, which can be found at www.wiley.com/go/jobreadygo. The data is in the file reviews_Digital_Music_5.json.gz and will need to be extracted.

NOTE You must extract the downloaded file before you can use it. In macOS or Linux, open the file and it will extract automatically. For Windows users, we recommend using 7-zip to extract the file; you'll find 7-zip at www.7-zip.org.

This file is in a modified JSON format. If you open the ...