GKE 1.30 control plane upgrade

[fgke]

My production control plan experienced a forced upgrade to GKE 1.30 this morning during the maint window.  I have the cluster configured with "Target release channel: No channel (not recommended)".  It believe it was running GKE 1.27 at the time.

However, the unexpected part of this is that one of the node pools was also upgraded to 1.30 as well despite having Auto Upgrade: Disabled.  This is the first time I've see a forced upgrade of node pools after a control plane upgrade.

Did any else experience a similar event?

[jayeshmahajan]

As per my understanding there are scenarios where Google Cloud may force an upgrade, particularly when moving between versions with breaking changes, security patches, or aligning with supported versions even thought you may have selected "Target release channel: No channel (not recommended)"

[nmagcalengjr]

Hi @fgke,

Welcome to Google Cloud Community!

When a version reaches end-of-life (EOL), nodes are automatically upgraded. However, relying on forced node upgrades for production environments isn't recommended. 

In certain cases, node upgrades might fail, or the node version can lag significantly behind the control plane, leading to a degraded cluster state. If you're using the 'no channel' option, it's essential to monitor the release notes for forced upgrades to plan accordingly and take the necessary steps.

The GKE team periodically upgrades your cluster control plane automatically, regardless of whether it is enrolled in a release channel or not. These upgrades move the control plane to newer stable versions of Kubernetes.

GKE schedules automatic upgrades for clusters from one minor version to the next supported minor version before the minor version reaches the end of support. The timing of this upgrade depends on the schedule for the cluster's release channel. For more details, see Estimated schedule for release channels.

In addition, GKE auto-upgrades the cluster to maintain its operability, compliance, and ensure it remains performant, available, and secure.

Note: GKE previously used the term “end of life” to refer to the “end of standard support” date. With the introduction of extended support, we updated this term to reflect the multiple periods of support for a GKE minor version. 

For more information, you can visit GKE Release Notes (No channel) for your reference.

I hope the above information is helpful.

[garisingh]

We definitely would not have gone from 1.27 to 1.30 as we don't skip versions for the control plane.  It was probably 1.29 to 1.30 as we started automatic upgrades for 1.29 to 1.30 on Sept 26.

Are you sure auto-upgrade is disabled on all of your node pools?