Seeing latency for http calls via external and internal application loadbalancers

[srinivasa458]

HI Colleagues, we are seeing high latency in the application calls from internet through external application loadbalancer through internal application loadbalancer to VMs.

All the VM metrics are under control. But we see latency reaching till 7 sec when we increased volume of connnections.
Does GCP ALbs choke during load testing conditions?

backend latency is also in control from both LBs to their respective backends. But still we see latency during medium volume conditions.

[Marvin_Lucero]

Hi @srinivasa458 ,

From my understanding, ALB is designed to handle a large number of connections, however, they are still affected by network congestions and latency (backend). Now, when you increase the volume of connections, the load balancer may need to distribute the load across more backend instances. More connection can result to increased latency since the LB tries to look for the best backend instance that will handle the request.

The following tips may improve the network latency:

1. You can increase the number of backend instances (horizontal scaling). As the incoming request increases, the additional instances will handle the load. You can check the autoscaling feature upon creating MIGs. 
2. Consider using a CDN as it will help reduce latency by caching content closer to the user. 
3. Try using Global Load Balancer. This can help distribute traffic more efficiently across multiple regions. This can help reduce latency by ensuring that requests are routed to the closest backend instance.

Let me know if this helps.