Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

How to solve 502 Error, Bad Gateway and GKE LoadBalancer

@Savvy 
@Marvin_Lucero 

Hi All,

Im not able to solve below 2 error can anyone have idea on it

Im using GKE Standard Cluster and we have mutplie service deploying in gke along with ingress 

Release Channel :- 1.28.10-gke.1075001

Ingress is configure

mohdrashid1_0-1721043872347.png

There is no issue in the code as developer said 100% correct it maybe issue from Ingress and LB, some configuration issue for which i dont have any idea

along with route, LB with target proxies, forwarding rules, backend services and TLS certificate all these are Active state and working.

 

 

mohdrashid1_1-1721042435002.png

mohdrashid1_0-1721043167090.png

This CPU and Memory usage for 30days showing

mohdrashid1_0-1721043785647.png

 

Error 1 

```
{@type: type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry, backendTargetProjectNumber: projects/983766048899, remoteIp: 147.185.132.94, statusDetails: invalid_request_headers}
```
Error 2

 

```

Error updating field details with AI service: https://xxxx.xxx.dev/api-analyzer/fetch-api-spec AxiosError: Request failed with status code 502
```
Here is screenshot of 502 error

 

mohdrashid1_0-1721042943402.png

Here is my yaml file which i used to deploy into gke standard cluster

```

apiVersion: apps/v1
kind: Deployment
metadata:
name: xxxxxxxxx
namespace: xxxxx
spec:
replicas: 2
selector:
matchLabels:
app: pxxxxxxxxxx
template:
metadata:
labels:
app: xxxxxxxxx
spec:
containers:
- name: xxxxxxxxxx
image: us-south1-docker.pkg.dev/xxxxxxxxxxxxxxxxxxxx:2024-07-04-1021
imagePullPolicy: Always
ports:
- containerPort: 8080
readinessProbe:
httpGet:
path: /ping
port: 8080
initialDelaySeconds: 30
periodSeconds: 5
timeoutSeconds: 10
livenessProbe:
httpGet:
path: /ping
port: 8080
initialDelaySeconds: 30
periodSeconds: 5
timeoutSeconds: 10
startupProbe:
httpGet:
path: /ping
port: 8080
initialDelaySeconds: 30
failureThreshold: 0
periodSeconds: 10
env:
- name: "DASHBOARD_API"
- name: "xxxxxxxxxx"
- name: "xxxxx"
- name: "xxxxxxxx"
resources:
requests: # Minimum amount of resources requested
cpu: 500m
memory: 512M
limits: # Maximum amount of resources requested
cpu: 1000m
memory: 1536M
---
apiVersion: v1
kind: Service
metadata:
name: xxxxxxxxxx
namespace: xxxxx
annotations:
labels:
app: xxxxxxxx
spec:
type: ClusterIP
selector:
app: xxxxxx
ports:
- port: 8080
targetPort: 8080
protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-xxxxxxxxx
namespace: xxxxxxxx
annotations:
ingressClassName: "gce"
kubernetes.io/ingress.global-static-ip-name: ingress-xxxxxxxxx
networking.gke.io/managed-certificates: managed-cert-xxxxx
spec:
rules:
- host: "test.cloudnetwork.in"
http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: xxxxxxxxxxx
port:
number: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-xxxxxxx
namespace: xxxxxx
annotations:
kubernetes.io/ingress.global-static-ip-name: ingress-exxxxxxxx
networking.gke.io/managed-certificates: managed-cert-xxxxxxx
ingressClassName: "gce"
spec:
defaultBackend:
service:
name: xxxxxxxx
port:
number: 8080
---
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: managed-cert-engine
namespace: xxxxxxxx
spec:
domains:
- test.cloudnetwork.in
---
# this optional purpose used
# apiVersion: cloud.google.com/v1
# kind: BackendConfig
# metadata:
# name: xxxxxxx
# namespace: xxxxx
# spec:
# healthCheck:
# timeoutSec: 1
# type: HTTP
# requestPath: /ping
# port: 8080

```
just here sharing small logs of that services, if need i can send u more logs
mohdrashid1_0-1721044062603.png

kubectl logs l7-default-backend-6484dd554-glcjw -n kube-system

```
I0714 01:34:20.325713 1 server-with-metrics.go:66] Default 404 server is running with GOMAXPROCS(4) on l7-default-backend-6484dd554-glcjw:8080
I0714 02:34:20.335582 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 03:34:20.344414 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 04:34:20.352695 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 05:34:20.361231 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 06:34:20.369615 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 07:34:20.370879 1 server-with-metrics.go:108] No connection requests received for 1 hour
I0714 08:34:20.379757 1 server-with-metrics.go:108] No connection requests received for 1 hour
```

1 1 667
1 REPLY 1

i got another error :- ERROR  [resource.labels.containerName: xxxx-xxxx] status: 504,
ERROR  [resource.labels.containerName: xxxx-xxxx] statusText: 'Gateway Time-out',
and 
ERROR  [resource.labels.containerName: xxxx-xxxx] status: 502,
ERROR  [resource.labels.containerName: xxxx-xxxx] statusText: 'Bad Gateway',

Top Labels in this Space