This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Cloud Deploy to GKE cluster with authorized networks

Posted on 06-05-2024 05:48 PM
jsok_algolia
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Hi,

I'm attempting to use Cloud Deploy to deploy to a GKE target (a PSC enabled cluster) which has a public endpoint with control plane authorized networks configured.

I created a Cloud Build worker pool but it is unable to access the GKE control plane - not unexpected. The worker pool VPC and the GKE control plane cannot be transitively peered. Flipping my Cloud Deploy GKE target between public and private IP doesn't help either.

The solution which seems to be recommended in this situation is to use https://cloud.google.com/build/docs/private-pools/accessing-private-gke-clusters-with-cloud-build-pr... so the worker pool can transparently access the GKE private endpoint. However looking at the Cloud VPN costs to achieve this it a bit outrageous, spending in excess of $500 USD a month to run a VPN just for Cloud Deploy is frankly not a solution. I'd need to do this for dozens of clusters too!

The workaround I've seen discussed is to deploy a bastion VM in my VPC and use it to proxy requests from my Cloud Build worker pool to the GKE control plane. This would be fine for my use case however what's lacking in Cloud Deploy's capability is the ability to customize the Cloud Build execution config used when deploying, i.e. at a minimum I'd need to be able to inject an HTTPS_PROXY env var.

The other solution would be to write a Cloud Deploy Custom Target and essentially re-implement what the GKE target is doing with the ability to proxy requests.

Putting aside the technical details, none of these solutions are exactly pleasant, easy or managed. I also don't think deploying to a private GKE cluster should be this difficult (whether via the private or public endpoint)! Have I missed something obvious here? Is nobody else facing this issue?

Appreciate any help, thanks!

1 2 850
Topic Labels
2 REPLIES 2

Posted on --/--/---- --:-- AM
mark_1233
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Not here with any solutions I'm afraid, but coming across the same issue. Google seem to like to promote all these integrations but they're never quite what they seem. I was hoping it would be more straightforward and out-of-the-box than using Spinnaker or Concourse, but unfortunately this isn't the case. I don't mind committing to more cloud specific tooling but only if it makes my life easier and doesn't come at an extortionate additional cost.

0 Likes

Posted on --/--/---- --:-- AM
jsok_algolia
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Looking at the latest Cloud Deploy release notes, it appears that we have a few alternatives to this problem now.

  1. Deploying using a Proxy is an new option: https://cloud.google.com/deploy/docs/gke-targets#deploy_using_a_proxy
  2. Cloud Build private pools authenticating to private GKE clusters using Identity Service for GKE: https://cloud.google.com/build/docs/private-pools/access-private-gke-clusters-identity-service

Option 2 looks better as you avoid the need to run a proxy, but the guide is fairly long and complicated. Will have to give it a go!

Top Solution Authors
View all