Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

How to config a gke-l7-rilb Gateway for HTTPs

Google documentation doesn't clarify how to do it, but my goal is to expose an App deployed in K8s into a VPC only, but could be seen by an user that uses a VPN in another project, using HTTPs protocol.
Context:
- Cluster is deployed in Project A, but in its corresponding VPC network, in this case, Project B.
- VPN server is deployed in Project C.
- I've successfully configured with HTTP and got access using the VPN client, but when I try to configure it for HTTPs, doesn't work.
- In console, Gateway is healthy, but ILB backend services are not.
- Firewall rules configured to allow backend services to hit pods in their respectives ports (8501 & 443).
- Self-signed certificate saved as K8s secret.
- Private DNS managed Zone and DNS A Record deployed and configured to its corresponding static ip address.

Current config (Terraform):

 

 

resource "kubernetes_service" "this" {
  metadata {
    name      = var.service_name
    namespace = kubernetes_namespace_v1.this.metadata[0].name
  }
  spec {
    type = "NodePort"
    selector = {
      app = var.service_name
    }

    port {
      name         = "port-http"
      protocol     = "TCP"
      port         = var.service_port
      target_port  = var.service_target_port
      app_protocol = "HTTP"
    }

    port {
      name         = "port-https"
      protocol     = "TCP"
      port         = 443
      target_port  = var.service_target_port
      app_protocol = "HTTPS"
    }

  }
  provider = kubernetes.gke
}

resource "kubernetes_manifest" "gateway" {
  manifest = yamldecode(
    <<-YAML
      kind: Gateway
      apiVersion: gateway.networking.k8s.io/v1beta1
      metadata:
        name: ${var.service_name}-internal-http
        namespace: ${kubernetes_namespace_v1.this.metadata[0].name}
      spec:
        gatewayClassName: gke-l7-rilb
        listeners:
        - name: http-listener
          protocol: HTTP
          port: 80
        - name: https-listener
          protocol: HTTPS
          port: 443
          hostnames:
          - "${var.service_name}.myapp.com"
          tls:
            mode: Terminate
            certificateRefs:
            - name: ${kubernetes_secret.tls_certs.metadata[0].name}
              kind: Secret
              group: ""
        addresses:
        - type: NamedAddress
          value: ${google_compute_address.static_ip_address.name}
    YAML
  )
  provider = kubernetes.gke
}

resource "kubernetes_manifest" "health_check_policy_https" {
  manifest = yamldecode(
    <<-YAML
      apiVersion: networking.gke.io/v1
      kind: HealthCheckPolicy
      metadata:
        name: ${var.service_name}-healthcheck-https
        namespace: ${kubernetes_namespace_v1.this.metadata[0].name}
      spec:
        default:
          checkIntervalSec: 10
          timeoutSec: 5
          healthyThreshold: 3
          unhealthyThreshold: 3
          logConfig:
            enabled: true
          config:
            type: HTTPS
            httpsHealthCheck:
              port: ${var.service_target_port}
              requestPath: /
        targetRef:
          group: ""
          kind: Service
          name: ${var.service_name}
    YAML
  )
  provider = kubernetes.gke
}

resource "kubernetes_manifest" "httproute" {
  manifest = yamldecode(
    <<-YAML
      kind: HTTPRoute
      apiVersion: gateway.networking.k8s.io/v1beta1
      metadata:
        name: ${var.service_name}-httproute
        namespace: ${kubernetes_namespace_v1.this.metadata[0].name}
        labels: 
          gateway: ${kubernetes_manifest.gateway.object.metadata.name}
      spec:
        parentRefs:
        - kind: Gateway
          name: ${kubernetes_manifest.gateway.object.metadata.name}
        hostnames:
        - "${var.service_name}.myapp.com"
        rules:
        - backendRefs:
          - name: ${kubernetes_service.this.metadata[0].name}
            port: ${var.service_port}

    YAML
  )
  provider = kubernetes.gke
}

resource "kubernetes_manifest" "gateway_policy" {
  manifest = yamldecode(
    <<-YAML
      apiVersion: networking.gke.io/v1
      kind: GCPGatewayPolicy
      metadata:
        name: ${var.service_name}-gateway-policy
        namespace: ${kubernetes_namespace_v1.this.metadata[0].name}
      spec:
        default:
          allowGlobalAccess: true
        targetRef:
          group: gateway.networking.k8s.io
          kind: Gateway
          name: ${kubernetes_manifest.gateway.object.metadata.name}
    YAML
  )
  provider = kubernetes.gke
}

 

 

 

I've looked in a lot of blogs, forums and sites without a clear answer. Thanks in advance.

Solved Solved
0 3 368
1 ACCEPTED SOLUTION

I've recreated the whole setup with some differences and now is working fine. I added a complete setup using cert-manager with a google issuer, so the certificates are renow automatically, and saved in a TLS secret using this annotation:

          tls:
            mode: Terminate
            certificateRefs:
            - name: ${local.cert_secret_name}

The firewall was updated, maybe the issue was there because I found tagged rules without an available target. And finnally I kept the healthcheck in HTTP. I didn't need and HTTPS healthcheck for this usecase.
PD: If your app will serve HTTP only, the k8s service just need a HTTP port, because the LB will struggle to encrypt/decrypt the request between the service and itself, resulting in errors.
Recomendation: Clarify how to set Firewall Rules in the particular case of setting Gateways, specially Regional Internal Gateways, documentation and references may be confusing.

View solution in original post

3 REPLIES 3

Hi, @FNavarro.

As far as, I know that the gke-l7-rilb does not support using self-signed certificates, you need to use a trusted certificate issued by a recognized Certificate Authority (CA) to configure HTTPS on your load balancer.Have you had a chance to try that as well?

Regards,
Mokit

Yes, I attached a certificate using Google provider, but with the same result. Backend services are still unhealthy.

 

tls:
   mode: Terminate
   options:
      networking.gke.io/pre-shared-certs:${google_compute_ssl_certificate.this.name_prefix}

 

 

I've recreated the whole setup with some differences and now is working fine. I added a complete setup using cert-manager with a google issuer, so the certificates are renow automatically, and saved in a TLS secret using this annotation:

          tls:
            mode: Terminate
            certificateRefs:
            - name: ${local.cert_secret_name}

The firewall was updated, maybe the issue was there because I found tagged rules without an available target. And finnally I kept the healthcheck in HTTP. I didn't need and HTTPS healthcheck for this usecase.
PD: If your app will serve HTTP only, the k8s service just need a HTTP port, because the LB will struggle to encrypt/decrypt the request between the service and itself, resulting in errors.
Recomendation: Clarify how to set Firewall Rules in the particular case of setting Gateways, specially Regional Internal Gateways, documentation and references may be confusing.

Top Labels in this Space