Cloud Run instance doesn't respond

[mudasir]

Hi,
I'm using Cloud Tasks to dispatch jobs to a Cloud Run instance. But since few days I'm encountering a weird behavior from the Cloud Run instance. Every day when the Cloud Tasks starts dispatching the requests the Cloud Run instance wouldn't respond and requests would get timed out. Nothing appears in the logs (no errors whatsoever),  just timeout errors of the requests from Cloud Tasks. After rebuilding the Cloud Run instance from Cloud Build when the new build gets deployed the Cloud Run instance starts behaving normally. I don't change anything in the code believe me and it starts serving the request from Cloud Tasks. This is weird, and I have no idea why would it happen, if it was a build issue then it wouldn't behave normally after rebuild.

Has anyone else encountered a similar issue? Would appreciate any help in this regard.

Thanks!

[christianpaula]

Hi @mudasir,

Welcome to Google Cloud Community!

Cloud Run instances not responding to Cloud Tasks can be caused by resource exhaustion, code bugs, or Cloud Run infrastructure issues.

To troubleshoot the issue, check the Cloud Run metrics and logs. If you are unable to find any errors, try rebuilding and redeploying the service. If you are still having problems, contact the Google Cloud support team for assistance.

To help prevent this issue from happening, use efficient coding practices, test your code thoroughly before deploying it to production, monitor your Cloud Run services regularly, and have a rollback plan in place.

[mudasir · Accepted solution]

Hi @christianpaula,
I think for resource exhaustion we usually do get errors in metrics (like I get when memory is fully used) and code bugs will also show up in logs, and for my case I don't see any errors, but the instance would cease to respond and only after rebuilding it works perfectly fine. And this would happen every day. I think I'll need to contact support for this one 👍

Thanks!

[monikadobreva]

Hi @mudasir,

Did you find what was causing the issue? I have the same problem and this is the only place I could find something about it.

Thank you for your response in advance!

[mudasir]

Hi @monikadobreva,

Wasn't able to find the root cause, but I redeployed the instance with old config and everything started working again. Then again changed the config to new config and this time everything worked. Seems to me it was a bug with Cloud Run itself.