Cloud Native Development with Google Cloud

Chapter 4. Preparing Google Cloud

In this chapter, you’ll embark on a journey that will fully equip you to work with Google Cloud. This includes account creation to the installation of essential tools designed to aid and enrich your experience in the project chapters that follow. You will also be introduced to the example project, which you will build into five separate projects in the subsequent chapters.

Create a Google Account

The first step toward using Google Cloud is creating an account. This can be accomplished by visiting the sign-up page and selecting the “Start free” button (Figure 4-1) in the upper right corner of the page.

Figure 4-1. Start free button

For those new to Google Cloud, a $300 free credit is available, which can be used over a period of 12 months to explore the platform’s services. This amount should cover the exercises in this book, allowing you to experiment without needing to spend money. In Chapter 15, there is information about the Google Innovators Plus program, an education subscription that also offers discounted Google Cloud credit.

Install the gcloud CLI

When signing into Google Cloud through a web browser, you will see the Google Cloud Console. Although the console provides access to nearly all Google Cloud’s services, this book focuses primarily on the gcloud CLI for its productivity and ease of explaining in writing.

The gcloud CLI is a command-line tool that provides interaction with Google Cloud Platform APIs. It is compatible with Windows, macOS, and Linux. Follow the steps outlined in the Google Cloud documentation for installation on your machine.

If, like me, you’re using a Mac with Homebrew, you can shortcut the installation process by running:

brew install google-cloud-sdk

Alternatively, you can use the Google Cloud Shell. This is a browser-based environment that is preinstalled with the gcloud CLI. You can access the Google Cloud Shell from the Google Cloud Console.

When signed in, look at the top right corner of the console and click the icon that looks like a command prompt labeled Activate Cloud Shell. This will give you an up-to-date gcloud CLI logged into your account, like the one shown in Figure 4-2.

Figure 4-2. Google Cloud Shell

If using Google Cloud Shell, you can skip the next two steps, which are only applicable when using the gcloud CLI from your own machine.

Update Your gcloud Client

Since Google updates gcloud components frequently, if you already have your gcloud client installed, it is often worth updating to the latest versions before starting work. You can do that with this command:

gcloud components update

If updates are available, you will be prompted for confirmation before installation.

Log In to gcloud

With an up-to-date gcloud client, you can now authenticate with your account using this command:

gcloud auth login

This will give you a link to open in your browser that will ask you to sign in with your Google account and then automatically authenticate your client. You will then be ready to start working with Google Cloud.

Set Your Default Region and Zone

Resources in Google Cloud are zonal, regional, or multiregional. That is, they apply to a single zone (location), multiple zones within a region, or distributed across multiple regions.

It is useful to specify defaults for your region and zone so that you do not have to specify them every time you create a resource. For example, if you are working in London (like me), you may want to specify the default region as europe-west2 and the default zone as europe-west2-a.

These commands set the default region and zone:

gcloud config set compute/region europe-west2
gcloud config set compute/zone europe-west2-a
gcloud config set run/region europe-west2

These settings are stored on your local machine; for example, on a Mac, in ~/.config/gcloud/configurations/config_default, they are set for your machine rather than for an account.

Tip

If you encounter a message like “API [compute.googleapis.com] not enabled on project [your-project-id],” it means that the specific service—in this case, the Google Compute Engine service—is not enabled for your project by default.

Not all services are enabled for new projects to avoid unnecessary usage and cost. When you create a new project, only a subset of services that are commonly used or required by other core services will be enabled automatically. This approach helps keep the project environment clean and reduces potential security risks.

Alternatively, you can use the gcloud init command to be guided through the process of setting up the gcloud CLI.

Create a Project

All Google Cloud Platform resources live in a project. A project is a container for resources that allows you to manage access and billing. Each project has a unique ID and a name. The name is unique to your account, but the ID is unique across all Google Cloud Platform accounts.

Tip

You will create several projects in this book. Rather than explaining how to set up a project each time, you will be referred back to this section.

I find it useful to store the PROJECT_ID in an environment variable so that I can easily reference it later on. As you move on to the projects, you will make a lot of use of this pattern:

export PROJECT_ID=[PROJECT_ID]

To create the new project, enter the following gcloud command:

gcloud projects create $PROJECT_ID

Then set the gcloud CLI to use the new project as the default:

gcloud config set project $PROJECT_ID

Similarly, if you have an environment variable named $REGION you could set the default region and zone like this:

gcloud config set compute/region $REGION
gcloud config set compute/zone $REGION-a
gcloud config set run/region $REGION

At any time, you can check your current project using:

gcloud config list project

If you ever want to set your current project to an environment variable again you use the output of the gcloud command like this:

export PROJECT_ID=$(gcloud config get project)

Enable Billing

By default, projects do not have a billing account enabled. This means you cannot create billable resources. Therefore, you need to enable billing by linking your project with an active billing account you have permission to use. With a new Google Cloud account, you should just have one billing account, and it should be active.

First, find ACCOUNT_ID for a billing account to use for the project by listing the billing accounts associated with your Google Cloud account. This is three blocks of six characters separated by hyphens:

gcloud beta billing accounts list

Now assign it to a BILLING_ACCOUNT_ID environment variable like you did with PROJECT_ID:

export BILLING_ACCOUNT_ID=[BILLING_ACCOUNT_ID]

You can then assign the billing account to the project using this command:

gcloud beta billing projects link $PROJECT_ID --billing-account $BILLING_ACCOUNT_ID

Note that here the gcloud command includes beta. You will see this occasionally. It just means it is a newer feature of the gcloud CLI. By the time you read this, the command may be fully integrated and beta will not be needed.

Checking Billing Is Enabled

You can check that billing is enabled on a project using the following command:

gcloud beta billing projects describe $PROJECT_ID --format="value(billingEnabled)"

This will return True if billing is enabled and False if it is not. If you see True, your project is now configured with a billing account and ready to use.

Tip

A nice feature of Google Cloud is that you can also remove the billing accounts from projects and be confident that you will not be billed. This will shut down the services in your project, so it is better to shut them down yourself first.

However, having worked with AWS, I still find I am billed a few dollars a month for forgotten resources I have not tracked down and removed, so I appreciate this Google Cloud feature.

The command to unlink a billing account from your project is:

gcloud beta billing projects unlink $PROJECT_ID

Doing More with the gcloud CLI

You will be using the gcloud CLI throughout this book. Although you can do nearly everything through the Google Cloud Console, as you get more familiar, the gcloud CLI you will find your productivity rise.

A good way to do this is to use the gcloud interactive environment by running the following command:

gcloud beta interactive

This gives you command completion and provides inline documentation, helping you to learn the commands and options available.

Key Google Cloud Concepts

Let’s take a moment to review some key concepts that will be used throughout this book.

Environment Files

Throughout the projects, values are stored in environment variables. This is a common pattern in the world of DevOps. It allows you to easily change values without having to change code. It also allows you to keep sensitive values out of your code and version control. Rather than setting the environment variables each time you start a new terminal session, you can store them in a file and then load them into your environment. This is what the .env file is for. Each project has a .env.template file that you can copy to .env and then fill in the values. At the root, there is also an environment file that holds common environment variables.

In each project, execute the set-env.sh script to set environment variables for you. This will apply the values in the .env file in the current directory together with the shared environment variables in the root .env file.

Enabling Services

Upon the initial creation of a project in Google Cloud, a number of services are not active by default. Attempting to utilize these services may result in an error. These services, however, can be easily activated using either the Google Cloud Console or the gcloud CLI. For instance, if you wish to activate the Cloud Run API, you can do so by executing the following command:

gcloud services enable run.googleapis.com

Whenever you use a service for the first time, you will see the command to enable it.

Identity and Access Management

Every Google Cloud Platform project has an identity and access management (IAM) policy. This policy specifies who has what type of access to which resources. Unlike what may be the case for an on-premises system, almost every resource or service in Google Cloud needs implicit permission to be accessed. This is a powerful and flexible system that allows you to control access to your resources.

In general, a principal (user) has roles that grant them permission to perform actions on resources.

Tip

It is important to understand that changes to IAM roles and permissions are eventually consistent and can take several minutes. This means that if you revoke a role or permission, it may still be available for a few minutes. Similarly, if you grant a role or permission, it may not be available for a few minutes.

Service Accounts

Service accounts are a special type of account used by applications acting as the principal to access Google Cloud services and resources. They are not intended for use by humans. They are used by the Google Cloud services themselves. As you start to join up services, you will be using service accounts to allow services to access other services.

While you can use a default service account for this, it is better to create a specific service account for each service. This allows you to control access to resources more granular. To follow this best practice, you will be using a service account for each service created in the projects.

Recommended Tools

Let’s review the tools you will be using throughout this book. These are not essential, but they will make your life easier, and they will be referenced in the projects.

Google Cloud Architecture Diagramming Tool

The Google Cloud architecture diagramming tool has been used to create the diagram architecture in this book. It supports all Google Cloud services and is free to use.

Command-Line Utilities

While much of this book assumes that you’re using a POSIX-compliant shell, such as those found on Linux or macOS systems, it’s not a hard requirement. You can still follow along using Windows PowerShell or the Windows Subsystem for Linux (WSL). Alternatively, the Cloud Shell, which I mentioned earlier, can also be used.

Let’s move on to some command-line utilities that you’ll find handy:

envsubst

envsubst is a handy command-line tool that substitutes environment variables in a file. It’s part of the gettext package and works across Windows, macOS, and Linux. Refer to the installation instructions.

Use envsubst for replacing environment variables in configuration files. This lets you maintain configuration settings for your gcloud environment separately from your code.

jq

jq is a command -line tool for parsing JSON. Many commands you will use have the option of outputting results in JSON format. Being able to extract information for use elsewhere is handy. jq is available for Windows, macOS, and Linux. Follow the instructions to install.

yq

yq is like jq but for YAML allowing extracting information from commands that output YAML. It is available for Windows, macOS, and Linux. Follow the instructions to install.

pack

In some projects, you will be using Cloud Native Buildpacks to build container images. The pack CLI is a tool for building container images using buildpacks. It is available for Windows, macOS, and Linux. Follow the instructions to install.

cURL

cURL is a command-line tool for sending HTTP requests. You will use this for testing HTTP endpoints from the command line. It is available for Windows, macOS, and Linux and is sometimes installed already. However, follow the instructions to install it on your local machine.

Apache Bench

Apache Bench is a tool for benchmarking web requests. It is a command-line tool that can be used to send a large number of requests to a web server and measure the response time.

If you are using a macOS, Apache Bench is already installed. If you are using Linux, you can install it using your package manager. If you are using Windows, you can install it using Chocolatey.

However, if you are using the Google Cloud Shell, Apache Bench is not installed by default. You can install it using the following command:

sudo apt-get install apache2-utils

Siege

Siege is a similar tool to Apache Bench, but Siege can provide log files of the test with more details, including the response of each request. Follow the instructions to install it.

Kubernetes

While the main Kubernetes command-line tool kubectl is provided as part of the Google Cloud SDK, there are a number of other tools that you will find useful:

k9s

k9s is a command-line tool for managing Kubernetes clusters. It is available for Windows, macOS, and Linux. Follow the instructions to install. It provides a command-line interface to Kubernetes that is much easier to use than the standard kubectl command, and it also looks cool, as shown in Figure 4-3.

Figure 4-3. k9s

kubectx and kubens

kubectx and kubens are a pair of command-line tools for managing Kubernetes clusters and namespaces. They are available on GitHub.

Terraform

Terraform is a tool for managing infrastructure as code that will be introduced toward the end of the book. It is available for Windows, macOS, and Linux. Follow the instructions to install. As well as Terraform itself, there are several other tools that you may find useful:

TFLint

TFLint is a tool for linting Terraform code. Follow the instructions to install.

TFSec

TFSec is a tool for checking Terraform code for security issues. Follow the instructions to install.

Infracost

Infracost is a tool for estimating the cost of Terraform code. It is useful for keeping track of potential Google Cloud expenses. Follow the instructions to install.

Introducing the Example Application

To better illustrate how Google Cloud services can be used, this book utilizes an example application named Skills Mapper.

Introducing Skills Mapper

The transition to a cloud native style of development involves more than just adopting new technologies and techniques. It also requires equipping your team with the necessary skills to build and support the applications. Understanding the existing knowledge and experience within your team, as well as the broader organization, can be invaluable in this transition.

While learning, it can be beneficial to identify peers with similar interests or those who are also learning. Knowing any existing experts in a particular topic can be a source of support and mentorship.

This is where our project, Skills Mapper, comes into play. Skills Mapper is a web application designed to track and map skills—tools, technologies, and techniques—that individuals are interested in, are currently learning, have experience in, or are planning to phase out. It maps these skills to a common ontology, providing an organized view of skill sets.

For individual users, Skills Mapper provides an API that can be utilized to generate a dynamic “living CV.” This CV can be displayed on a web page or incorporated into an online resume. For groups of users, Skills Mapper illustrates how their combination of skills compares to their peers, suggests what they should learn next, and provides insights into trending interests.

Within an organization or a community where Skills Mapper is in use, it serves as a tool to locate experts, construct job profiles, and suggest communities of practice. It can also support the planning of training, and study groups, or identify the skills individuals are seeking to acquire.

Throughout this book, the Skills Mapper project will be developed and scaled as a microservices architecture, evolving from a tool for individuals to an application that can efficiently serve thousands of users.

Skills Mapper Architecture

The Skills Mapper application consists of three microservices, each utilizing a different type of storage. The architecture of the application is displayed in Figure 4-4.

Figure 4-4. Skills Mapper architecture

Each microservice is responsible for a single function, with its capabilities exposed through a REST API:

Skill service (Chapter 6)

This service suggests skills from a list gathered from Stack Overflow.

Fact service (Chapter 7)

This service allows authenticated users to manage their skills.

Profile service (Chapter 8)

This service automatically creates a profile and provides authenticated users with profile information.

A common API exposes the microservices and, in turn, a user interface interacts with the API in Chapter 9.

In the background in Chapter 5, a utility function is used to refresh the list of tags weekly.

To ensure security, everything is placed behind a load balancer with a public IP address associated with a DNS name in Chapter 11.

Services Used

Here is a comprehensive but nonexhaustive list of the services you will be using throughout this book. The chapters in which each service is used are also shown in the table.

Service	Description	Chapters used
BigQuery	Data warehouse	Chapter 5
Cloud Functions	Serverless functions	Chapter 5
Cloud Run	Serverless containers	Chapters 6, 7, and 8
Cloud Storage	Object storage	Chapters 5, 8, and 9
Cloud Build	CI/CD	Chapter 12
Cloud Scheduler	Scheduled tasks	Chapter 5
Cloud Pub/Sub	Messaging service	Chapter 8
Cloud Firestore	NoSQL document database	Chapter 8
Cloud SQL	Relational database	Chapter 7
Cloud Spanner	Distributed relational database	Chapter 14
Identity Platform	Authentication provider	Chapter 7
Global HTTP Load Balancer	Load balancer	Chapter 11
Cloud Armor	Web application firewall	Chapter 11
GKE Autopilot	Managed Kubernetes	Chapter 14

Summary

Now that you’ve set up a project and installed the necessary tools, you’re fully prepared to begin building on Google Cloud Platform. You will start the journey in Chapter 5.