Help Understanding Endpoint Instantiation Issues (9/18/2024)

[drce]

Hi Google community,

I'm seeking help understanding an issue I encountered today (between 4:00pm and 5:00pm EDT). Google was having trouble instantiating some firebase functions during that time, and the error in the logs stated:

"The request was aborted because there was no available instance. Additional troubleshooting documentation can be found at: https://cloud.google.com/functions/docs/troubleshooting#scalability"

I checked the Google Cloud Status Dashboard, but no issues were reported during that time. I also reviewed the troubleshooting documentation linked in the error message, but I’m still unclear on the root cause. What's confusing is that some of the endpoints affected had minInstances configured, so theoretically, there should have been at least one idle instance available.

Any insights or advice for diagnosing or preventing this in the future would be greatly appreciated!

Thanks in advance for your help.
Daniel

[policenauts · Accepted solution]

Edit: Google has acknowledged the issue: https://status.cloud.google.com/incidents/1yphfNLPHEnwJcWqwxbu

@drce  We also faced this issue today with a couple of our v1 Cloud Functions which had been running fine for years, us-central1, no minInstances configured. I posted in the in the GCP Community Slack and two others also reported similar.

We redeployed with a minInstance count and it appears to have helped for now, but are hoping for some kind of acknowledgment from Google that there was a bug today. 

[drce]

Thank you, @policenauts  for your reply.

[paladindivy]

We also faced a similar issue and are hoping for a RCA (root cause analysis) or better understanding of what happened so we can prevent this from happening. need some resolutions immediately. 

[drce]

Thank you, @paladindivy   for your reply.