Cloud Run service high latency

[ramesh1600]

We have an application with low latency requirements. Expectation is to have a response time between 2 to 10 milli seconds per HTTPS request.

Load Balancer + Instance Group : 2 milli seconds

When the application is deployed in an instance group with a HTTPS Load balancer as front end, we are getting 2 milli second response time.

Cloud Run service : 20 milli seconds

However when we deploy the same container in a Cloud Run service, the response time shoots up to 20 milli seconds. CPU was always allocated with minimum 1 instance to avoid warm up issues. Second generation execution environment.

Actually we were expecting on-par or better performance in Cloud Run due to the self managed Load Balancer and Service integration. However the latency is very high when compared to the instance group approach.

Is there a way I can further reduce the latency in the Cloud Run service ?

Please note in both tests, the client machine (from where the HTTPS requests made) was kept in us-west1 where my Regional Load Balancer and Instance Group also reside. I can provide any additional information if required.

[juliadeanne · Accepted solution]

Hello @ramesh1600 ,

Welcome to Google Cloud Community!

I noticed you've also posted on the Reddit forum about experiencing high latency with the Cloud Run service. It's great to see community engagement and the exchange of insights.

One community member commented on your post, and they are correct—achieving a latency of 20ms is indeed quite impressive already.

According to our documentation, the lowest minimum latency achievable with external load balancing is 123ms. For more details, you can refer to our guide on optimizing application latency with external Load Balancing . 

If you're aiming to further optimize your external Application Load Balancer and strive towards your latency goals, integrating with Cloud CDN can be a beneficial step. More information about Cloud CDN can be found here. 

Moreover, our documentation offers comprehensive insights on optimizing external Application Load Balancers. Feel free to explore the documentation for more strategies and tips.