Admission controller and static IP for GKE

[fearlsgroove]

In a GKE cluster, i'm trying to create a LoadBalancerIP with a static/reserved external IP address. This works if I let it create an ephemeral IP, but I want to give it a static IP since it will be publicly durable.

If i try to specify LoadBalancerIP on the service i get:

Error: failed to create resource: services "myapp-traefik" is forbidden: Use of external IPs is denied by admission control

I can't find any way to override the admission controller.

apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/neg: '{"ingress":true}'
    meta.helm.sh/release-name: myapp
    meta.helm.sh/release-namespace: myapp
  creationTimestamp: "2022-12-07T00:26:13Z"
  finalizers:
  - service.kubernetes.io/load-balancer-cleanup
  labels:
    app.kubernetes.io/instance: myapp-myapp
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: traefik
    helm.sh/chart: traefik-20.6.0
  name: myapp-traefik
  namespace: myapp
  resourceVersion: "118469"
  uid: b17c77b6-dc84-4e7a-9738-929fbdab6f59
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 10.76.3.237
  clusterIPs:
  - 10.76.3.237
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: web
    nodePort: 32640
    port: 80
    protocol: TCP
    targetPort: web
  - name: websecure
    nodePort: 30918
    port: 443
    protocol: TCP
    targetPort: websecure
  selector:
    app.kubernetes.io/instance: myapp-myapp
    app.kubernetes.io/name: traefik
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: xx.xx.xxx.xxx

[garisingh]

Are you specifying spec.loadBalancerIP in your deployment manifest?

[fearlsgroove]

Thanks for the reply. Yes I'm specifying the loadBalancerIP in the service. It's a global, static reserved IP. The admission controller is preventing the deploy from happening. It's this specifically:
https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#denyserviceexternalip...

I had not heard of admission controllers before now. It's a brand new cluster running GKE 1.23.12-gke100. There's no way to disable or adjust admission controller plugins in GKE as far as I can see, but it appears all the existing docs on GKE are wrong as of now. A bug?

[garisingh]

Yeah ... I realized that's the admission controller giving the error, but that should not be applicable for using a static IP on a Service of type LoadBalancer.  I'll run a test myself as well.  In the meantime, you can update your cluster with this flag:  https://cloud.google.com/sdk/gcloud/reference/container/clusters/update#--enable-service-externalips

[garisingh · Accepted solution]

On an Autopilot cluster running 1.23.12-gke.100, I deployed the following Service:

apiVersion: "v1"
kind: "Service"
metadata:
  name: "ping-demo-static-service"
  labels:
    app: "ping-demo-static"
spec:
  ports:
  - protocol: "TCP"
    port: 80
    targetPort: 8080
  selector:
    app: "ping-demo-static"
  type: "LoadBalancer"
  loadBalancerIP: "XXX.XXX.XXX.XXX"

and everything worked fine.

[fearlsgroove]

Thanks for humoring me. Came back to this today to try to get it sorted. I'm deploying traefik via helm, and while I thought I was correctly checking the output of helm via dry run and looking at the resulting spec, I must have been setting the wrong value in the helm chart to cause the issue. Tried again today and it seems to be working as expected.

For reference the correct setting for the official traefik helm chart is:

service.spec.loadBalancerIP

[Harsh_Manvar]

You need to have the field in service section

loadBalancerIP: "YOUR.IP.ADDRESS.HERE"  

 Refer for more details : 

https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#exposing...

Also hope your static IP and cluster in the proper region not in different regions.

[fearlsgroove]

Thanks for the reply -- please see the other comment. I am specifying loadBalancerIP on the service, the IP is a global static IP, and the error in OP is the result. This admission controller plugin is the source of the error:
https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#denyserviceexternalip...