exceeded quota: gke-resource-quotas, requested: count/ingresses.networking.io=1

[Master786]

Hi Team,

While deploying our SAS Viya application , we are facing the below quota exceeded issue. Could you please suggest if we can edit the quota for this resource using the quotas page in GCP. If yes by which filter we can locate this resource.

If not using console then how the limit can be increased.

Error from server (Forbidden): error when creating "site.yaml": ingresses.networking.k8s.io "sas-themsigner-app" is forbidden: exceeded quota: gke-resource-quotas, requested: count/ingresses.networking.io=1, used: count/ingresses.networking.k8s.io=100, limited: count/ingresses.networking.k8s.io=100

Thanks in advance.

Regards,

Sharique

[glen_yu]

GKE resource quotas are applied automatically and can't be removed for clusters under a certain size.

https://cloud.google.com/kubernetes-engine/quotas#resource_quotas

I know the link above says it's for "clusters with 100 nodes or fewer", but I think it's much less than that.  How many nodes are in that cluster where you're getting the resource quota limit error?

EDIT: if you want to not have the quota limit, I think you'll need to scale the number of nodes past a certain number (I'm not sure what that number is, but "100" is awfully high -- it might be a typo in the doc)

[glen_yu]

I did some more investigating and built myself a zonal GKE cluster with a single (1) node in its node pool and here's its resourcequota:

apiVersion: v1
kind: ResourceQuota
metadata:
  creationTimestamp: "2021-08-25T14:12:03Z" 
  name: gke-resource-quotas
  namespace: default
  resourceVersion: "1510"
  uid: ec48abd3-f058-4441-bd0a-f2257421fe4d
spec:
  hard:
    count/ingresses.extensions: "100"
    count/ingresses.networking.k8s.io: "100"
    count/jobs.batch: 5k
    pods: "1500"
    services: "500"
status:
  hard:
    count/ingresses.extensions: "100"
    count/ingresses.networking.k8s.io: "100"
    count/jobs.batch: 5k
    pods: "1500"
    services: "500"
  used:
    count/ingresses.extensions: "0"
    count/ingresses.networking.k8s.io: "0"
    count/jobs.batch: "0"
    pods: "0"
    services: "1"

Then I scaled it up to 11 nodes and here are the new resourcequota numbers:

apiVersion: v1
kind: ResourceQuota
metadata:
  creationTimestamp: "2021-08-25T14:12:03Z"
  name: gke-resource-quotas
  namespace: default
  resourceVersion: "8767"
  uid: ec48abd3-f058-4441-bd0a-f2257421fe4d
spec:
  hard:
    count/ingresses.extensions: 100k
    count/ingresses.networking.k8s.io: 100k
    count/jobs.batch: 50k
    pods: 15k
    services: 5k
status:
  hard:
    count/ingresses.extensions: 100k
    count/ingresses.networking.k8s.io: 100k
    count/jobs.batch: 50k
    pods: 15k
    services: 5k
  used:
    count/ingresses.networking.k8s.io: "0"
    count/jobs.batch: "0"
    pods: "0"
    services: "1"

Just going from 1 to 11 nodes increased the ingress limit 1000x, so maybe what you're looking for is somewhere in between, but I think at the very least you'll have to increase your node count.