This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Configuring GKE rollout sequencing

Posted yesterday
and filed in cloud-community-blog
bhardwaju
Staff

Overview

You can manage the order of automatic cluster upgrades across Google Kubernetes Engine (GKE) clusters in multiple environments using rollout sequencing. For example, you can qualify a new version in pre-production clusters before upgrading production clusters.

When GKE upgrades a cluster, first the control plane is upgraded, then the nodes are upgraded. In a rollout sequence, clusters are still upgraded using this process, but you also control the order in which groups (fleets or scopes) of clusters are upgraded, and you specify a soak time to choose for how long GKE pauses before upgrades proceed from one group to the next group.

In this lab, you learn how to create and configure rollout sequences using fleets.You also learn how to define soak time to control upgrade rollout speed and monitor and verify upgrades across different environments.

image7.pngObjectives

  • Create and Register GKE clusters to the fleet.
  • Create and configure rollout sequences using fleets.
  • Define soak time to control upgrade rollout speed.
  • Monitor and verify upgrades across different environments.

Setup and requirements

Before you click the Start Lab button:

  • Read these instructions.
  • Labs are timed, and you can’t pause them.
  • The timer, which starts when you click Start Lab, shows how long Google Cloud resources are available to you.
  • This Qwiklabs hands-on lab lets you perform the lab activities yourself in a real cloud environment, not in a simulation or demo environment.
  • It gives you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.
What you need

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
  • Time to complete the lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab.

Note: If you are using a Pixelbook, open an Incognito window to run this lab.

How to start your lab and sign in to the Google Cloud Console
  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is a panel populated with the temporary credentials that you must use for this lab.
    image4.png
  2. Copy the username, and then click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.image6.png         Tip: Open the tabs in separate windows, side-by-side.
    If you see the Choose an account page, click Use Another Account.
    image2.png
  3. In the Sign in page, paste the username that you copied from the Connection Details panel. Then copy and paste the password.
    Important: You must use the credentials from the Connection Details panel. Do not use your Qwiklabs credentials. If you have your own Google Cloud account, do not use it for this lab (avoids incurring charges).
  4. Click through the subsequent pages:
    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.image3.pngClick Continue.

image5.pngIt takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

image1.pnggcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

You can list the active account name with this command:

gcloud auth list

(Output)

Credentialed accounts:
 - <myaccount>@<mydomain>.com (active)

(Example output)

Credentialed accounts:
 - google1623327_student@qwiklabs.net

You can list the project ID with this command:

gcloud config list project

(Output)

[core]
project = <project_ID>

(Example output)

[core]
project = qwiklabs-gcp-44776a13dea667a6

For full documentation of gcloud see the gcloud command-line tool overview.

Execution

STEP 1: Run the following command to create two autopilot clusters Test and Stage.
gcloud container clusters create-auto CLUSTER_NAME \
    --location=LOCATION \
    --project=PROJECT_ID
    --enable-fleet

Replace the following:

  • [**]: the name of your new Autopilot cluster.
  • [**]: the region for your cluster.
  • [**]: your project ID.
STEP 2: To register the GKE clusters to the project’s fleet, run the below command.
gcloud container clusters update CLUSTER_NAME --enable-fleet

Replace the following:

  • [**]: the name of your existing Autopilot cluster created in Step 1.
STEP 3: Enable the below API(s)
gcloud services enable \
   --project=FLEET_HOST_PROJECT_ID \
   container.googleapis.com \
   gkeconnect.googleapis.com \
   gkehub.googleapis.com \
   cloudresourcemanager.googleapis.com \
   iam.googleapis.com
STEP 4: Grant Access Permissions
gcloud projects add-iam-policy-binding FLEET_HOST_PROJECT_ID \
   --member user:GCP_EMAIL_ADDRESS \
   --role=roles/owner

Replace the following:

  • FLEET_HOST_PROJECT_ID: with the ID of the first fleet's host project.
gcloud projects add-iam-policy-binding FLEET_HOST_PROJECT_ID \
   --member user:GCP_EMAIL_ADDRESS \
   --role=roles/container.admin

Replace the following:

  • FLEET_HOST_PROJECT_ID: with the ID of the first fleet's host project.
STEP 5: Create a rollout sequence

Create a rollout sequence:

  1. Set the soak time for the first fleet in the sequence: 
    gcloud container fleet clusterupgrade update \
    --default-upgrade-soaking=SOAK_TIME \
    --project=FIRST_FLEET_PROJECT_ID


    Replace [**] with the project ID of the fleet host project.

Set the upstream fleet and the soak time for the second fleet in the sequence:

gcloud container fleet clusterupgrade update \
    --upstream-fleet=FIRST_FLEET_PROJECT_ID \
    --default-upgrade-soaking=SOAK_TIME \
    --project=SECOND_FLEET_PROJECT_ID


Replace [**] with the project ID of the first fleet's host project, and [**] with the project ID of the fleet host project.

STEP 6: Check status of a rollout sequence
  1. Go to the Rollout Sequencing page in the Google Cloud console.
    Go to Rollout Sequencing
  2. View the sequence in the section Monitor your rollout sequence. If you don't see a rollout sequence, switch to a different rollout sequence, or create a rollout sequence if you haven't already done so.
How to use the console to monitor a rollout sequence

On this page, you can view the rollout sequence associated with your project's fleet. You can do the following to see the progress of a rollout sequence:

  • View the entire rollout sequence, or see the statuses of individual fleets and clusters within those fleets, as well as the soak time between fleets. You can also view the sequence where there is no active upgrade, if you want to check the configuration of the sequence.
  • Filter by upgrade type (control plane or node upgrade) and specific version (for example, 1.31.6-gke.500).

You can visually monitor your entire rollout sequence while GKE upgrades all the clusters in the sequence, qualifying a new version across environments before upgrading your production environment clusters. While monitoring, you can manage a rollout sequence with the gcloud CLI, making any changes as needed.

Use the gcloud CLI

Use these commands in the following sections to check on how upgrades are progressing in a rollout sequence. To learn more about what details are provided, see Status information for a rollout sequence

To run these commands, ensure that you have the required permissions for each fleet host project. For example, if the sequence has cross-project scopes in different fleets, you need permissions in each project to describe the sequence.

For the following commands, if you only need information about one fleet or scope in the sequence, replace the --show-linked-cluster-upgrade flag with --show-cluster-upgrade.

Fleets

Check the status of a fleet-based rollout sequence:

gcloud container fleet clusterupgrade describe \
    --show-linked-cluster-upgrade --project=FLEET_PROJECT_ID

Replace [**] with the project ID of the host project for any fleet in the sequence.

0 0 100
0 Likes