Failed to submit prompt Error message: "Internal error encountered." Status: 500 Error code: 500

[Jayakaran]

Hi all,

Created following a sample JSONL format,
Uploaded in the GCS bucket. gs://xxx/newsam.jsonl
we have created our own custom model using the curl command.

curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json; charset=utf-8" -d "@model-request.json"

The pipeline was successfully executed and deployed in the model registry.
Model name is : LLM

From the model registry, we have created the end point.
End point name :
sample1
Endpoint id: 59XXX079194845184

When we test the end point using the curl command, it throws "internal error encountered," error 500.

[kvandres]

Good day @Jayakaran,

There are several reasons why you are encountering this error, verify the following if it will solve your problem.

1. Verify if your custom model is trained using PyTorch, TensorFlow, XGBoost or scikit-learn, in a correct file format and exactly matches the filename format after you have uploaded them in GCS. Below are the correct format for model artifacts.

• TensorFlow SavedModel: saved_model.pb
• PyTorch: model.mar
• scikit-learn: model.joblib or model.pkl
• XGBoost: model.bst, model.joblib, or model.pkl

You can check this link fore more information:  https://cloud.google.com/vertex-ai/docs/model-registry/import-model#upload_model_artifacts_to

2. Verify if you are requesting to the correct API endpoint, if you are importing model here is the endpoint: 

POST https://$LOCATION_ID-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/models:upload

You can check this link for more information: https://cloud.google.com/vertex-ai/docs/model-registry/import-model#aiplatform_upload_model_sample-d...
Here is the endpoint when you are requesting a prediction: 

"https://$REGION-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$REGION/endpoints/$ENDPOINT_ID:predict"

 You can check this link for more information: https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/predict

but I highly recommend that you reach out to Google Cloud Support, if the issue persists after validating the solutions: https://cloud.google.com/support

Hope this helps!