Accessing the private gke cluster from cloud build

[gowtham3656]

Hi team, I have created a private gke cluster in gcp now i want deploy k8s files using cloud build and workerpool. Now i am getting error while accessing the gke cluster using cloud build. unable to connect xx.xx.xx.xx:443.io timeout any one can help to solve this error.

[comaro]

This issue often appears when the cluster is private, but gcloud still tries to use an external IP to dial. To avoid this, you could try adding the flag “--internal-ip” to the command.

[harshit18hkgm]

Can you explain more. In which command I specify  --internal-ip.

[alexmoore]

If you are using Cloud Build Private Pools, then it might be that there is a solution to this here that is worth trying:

https://cloud.google.com/architecture/accessing-private-gke-clusters-with-cloud-build-private-pools

The issue is that Cloud Build Private Pools reach into your VPC via VPC Peering, but additionally you reach the masters of a GKE Private cluster via VPC peering too.  So the issue is you need traffic from the Worker to come via peering to your VPC and then onwards via peering to the GKE master, but this would be transitive peering which is not supported and so it doesn't work.  

The solution I linked above addresses this constraint by using a second VPC for the Private Pools peering and then connecting it via a Cloud VPN connection into your VPC, at this point all the routing will work.

Hope that helps.

[jalvarez-cr]

The current approach seems to excessively complicate a task that should ideally be straightforward. Personally, I would even prefer the possibility of hosting my own workers in my virtual private cloud (VPC) instead of going through all these complexities.

[g_munish]

@alexmoore We shouldn't need to do this if the private GKE cluster has public endpoint enabled. We just need to whitelist the IP address. How to do this with private pool is unclear. The documentation doesn't clearly specify the steps. Could you help?

https://cloud.google.com/build/docs/private-pools/use-in-private-network#deploying-to-clusters

[gnubibi33]

Hello,

Get same disappointment about this setup 😞

Should be a security point to facilitate deployment on GKE with cloudbuild without to have controle plane open to allow 😞

It's not possible to use a subnet on the VPC for the private pool and not PSC ?

Thinking about stopping using cloudbuild and move to gitlab runner...a lot of work to migrate, but for security....