Error from Vertex AI Getting predictions from custom trained models

[jinmc]

I am trying to call an API to inference from a model I have uploaded to vertex AI.

I have tried three methods, and none worked so far.

At first, I was following a youtube from standford university, https://www.youtube.com/watch?v=fw6NMQrYc6w&t=3876s which uses ai platform.

1. I also tried that, but I think google is trying to get rid of AI platform, and although I succesfully uploaded the model, it doesn't allow me to make a new version, basically allows me nothing.

2. I tried to work this tutorial, https://codelabs.developers.google.com/vertex-p2p-predictions#5 

and it keeps complaining that my payload is above 1.5MB limit, but my image is only 49KB, so it's ridiculous. maybe something happened in this code, but it's from the tutorial, so the tutorial must be wrong then.

IMAGE_PATH = "test-image.jpg"
im = Image.open(IMAGE_PATH)
x_test = np.asarray(im).astype(np.float32).tolist()
endpoint.predict(instances=x_test).predictions

3. Last, I've been trying to call the API from the sample code,

https://github.com/googleapis/python-aiplatform/blob/main/samples/snippets/prediction_service/predic...

but it gives me a json format error.

I have referenced from this website to get the json format.

https://github.com/googleapis/python-aiplatform/blob/main/samples/snippets/prediction_service/predic... 

The error I am getting is as is : 

400 { "error": "Failed to process element: 0 key: instances of 'instances' list. Error: Invalid argument: JSON object: does not have named input: instances" }

The code that I have used is :

    encoded_content = base64.b64encode(image).decode("utf-8")
    instances = {"instances": {"image": {"b64": encoded_content}}, "key": "0"}

and the 400 error comes from API and the log from vertex AI is not very useful when debugging.

Honestly I have been struggling with this issue for days and in my opinion, this should not be this difficult. My experience with GCP and vertex AI is very disappointing and I'm considering to explore other options. Please let me know if any of you have any advices. Thanks

[Aris_O]

Hi @jinmc,

Welcome back to Google Cloud Community.

Based on analyzing the issues you've encountered here are some possible answer:

1. You may try referring to a more updated Vertex AI platform if you were utilizing an AI Platform-based lesson. For the Vertex AI documentation on GCP you may refer here: https://cloud.google.com/vertex-ai/docs.

2. Check the data type of your "x_test" variable if you are having problems with the payload size in the second technique you tried. The code you gave first turns the image into a NumPy array and then into a list of floats, but it doesn't display the shape of the list that results. The list might be significantly bigger than anticipated, which would push the payload above the cap. Before delivering the list to the endpoint, you can try printing the list's shape to check its size.

3. The third method's error indicates that the JSON payload you are submitting to the API is not in the proper format. Each instance of the JSON object in the payload should have a dictionary of input features. The code you supplied generates a nested dictionary with an "image" key and a dictionary with a single "instances" key. Consider making the payload a list of instances, where each instance is a dictionary with just one "image" entry.

Here is an example on how could you create the payload using a list of instances:

encoded_content = base64.b64encode(image).decode("utf-8")
instances = [{"image": {"b64": encoded_content}}]

This list of instances can then be added as the "instances" option in your API request.

Here are some references that might help you:
https://cloud.google.com/vertex-ai/docs/training-overview
https://cloud.google.com/vertex-ai/docs/tutorials

[jinmc]

I 

Hi @Aris_O  .

Thank you for quick reply. 

I have applied the change you have suggested and gives this error.

400 { "error": "Failed to process element: 0 key: image of 'instances' list. Error: Invalid argument: JSON object: does not have named input: image" }

Also, I am not training on vertex AI, I am just using vertex AI as a serving tool, 

so it might be a model issue. 

I might try another model. 

Again, thanks for your reply.

[jinmc · Accepted solution]

I actually solved this error by applying this document.

https://cloud.google.com/vertex-ai/docs/samples/aiplatform-predict-image-classification-sample.