Trying to use the Stable Diffusion model from Model Garden - getting 503 error

[SaiMadhavan]

Hi

I am trying to deploy and test the Model from Model Garden - Stable Diffusion v 2.1 - these are the steps i followed

Step-1 - Went to Model Garden and selected Stable Diffusion v2.1
Step-2 - View details and clicked Deploy
Step-3 - Followed all the default steps that are given in the deployment window
Step-4 - Model got deployed successfully
Step-5 - To test - created the endpoint and endpoint created successfully 
Step-6 - on the Test Dialog - i provided the input

{
"instances": [
{
"prompt": "sample txt to image description"
}
]
}

when i run the online predictions - the process runs and I get an error - 503 and the below error message

{
 "error": {
   "code": 503,
   "message": "Model server is not available. Please retry. endpoint_id: XXXXXXXX09120, deployed_model_id: XXXXXXXX1312",
   "status": "UNAVAILABLE"
 }
}

I went to the notebook and read that this error happens 

NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed after the above model deployment step succeeds and before you run the next step below. Otherwise you might see a ServiceUnavailable: 503 502:Bad Gateway error when you send requests to the endpoint.

In my case i am running the online predictions after deployment and endpoint creation is done and post that 5-10 minutes time period.

Any help will be appreciated 

[lsolatorio]

Hi @SaiMadhavan, 

The errors you have encountered seems to be server and browser related.The 503 code refers to unavailability of service, while 502 code refers to a bad gateway error that is usually caused by an invalid response from another server, these means that the server was not able to process the request possibly due to a system error on Google Cloud, an issue with the model deployment, or a problem with the service account. 

The things to try and check for such occurrences are:

• Waiting for a few minutes then trying again as the server might be temporarily unavailable at the time.
• Inspecting your Cloud Console as there might be errors related to your model server or deployment
• Model redeployment to force the server to reload your model
• Clearing your browser's cache as it can fix problems caused by the data stored in your browser's cache
• Try using a different device or browser as it might be an isolated case  

If the problem still persists, you can reach out to the Google Cloud support team for help. 

[SaiMadhavan]

Hi @lsolatorio - thanks for your response

I did tried running this from all 3 browsers - chrome, firefox and microsoft and same result. Not sure whether the issue is because i took the standard model from the Model Garden and try to use it as-is.

 The operation failed due to the following error(s):

Model server connection error. Please retry. endpoint_id: 1911632906283909120, deployed_model_id: 3443408734427021312

this is what i can see in the view issues/console dialog - nothing else.

Let me know

Thanks

[petecybriwsky]

Hi @SaiMadhavan -- did you ever figure out an answer to this issue? Running into the same issue with another off the shelf model and cannot for the life of me figure out why the service is unavailable. Have checked quotas, API enablement, etc.

Thank you!

[hiber-stefan]

I'm run into this trying to follow the
https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/...

Notebook.