how to add image to vertex ai agent conversations and have the agent analyze the image via api?

[danctk]

This is the service I am using:
https://cloud.google.com/dialogflow/vertex/docs/quick/api

I am able to create an agent, upload files as datastore/tools for the chatbot agent as reference. However, i want the users to send an image (in base64) and have the chatbot answer questions on that image. Is there a page in the documentation that shows us what the request body should look like? or is this even possible?

[ruthseki]

Hi @danctk,

Welcome to Google Cloud Community!

This page provides the request format for users to submit an image in Base64, enabling the chatbot to respond to questions about the image.

In the context of Vertex AI Agent Builder, the union field image with the imageBytes option indicates that the agent can handle image data in the form of byte strings. This capability allows the agent to process and understand images by receiving the image data encoded as a string of bytes.

I hope the above information is helpful.