Vertex AI agent with fine-tuned models

[robertomotta]

Hi,

i'm trying to create a vertex AI agent that uses a fine-tuned model.

I do not see in the web interface any way to use the fine-tuned model that i have created. Is it possible?

What i'm trying to do is: i have a pdf with a list of product, based on the conversation with the user i have to extract from the pdf the correct product requested.

Thank you

[ruthseki]

Hi @robertomotta,

Welcome to Google Cloud Community!

Yes, it is possible to use a fine-tuned model with Vertex AI. For further guidance, please refer to the following documentation:

• Models that support supervised fine-tuning
• How to tune LLMs in Generative AI Studio
• Build, tune, and deploy foundation models with Vertex AI

Here’s an example of fine tuning in Vertex AI:

• Fine-Tuning Large Language Models: How Vertex AI Takes LLMs to the Next Level

Additionally, you may consult the following release notes pertaining to the multimodal model that supports PDF files:

• April 9, 2024

Gemini 1.5 Pro (gemini-1.5-pro-preview-0409) is available in Preview. Gemini 1.5 Pro is a multimodal model that analyzes text, code, audio, PDF, video, and video with audio.

• May 14, 2024

Gemini 1.5 Flash (gemini-1.5-flash-preview-0514) is available in Preview. Gemini 1.5 Flash is a multimodal model designed for fast, high volume, cost-effective text generation and chat applications. It can analyze text, code, audio, PDF, video, and video with audio.

• June 11, 2024

You can upload media, such as PDF, MP4, WAV, and JPG files from Google Drive, when you send image, video, audio, and document prompt requests.

I hope the above information is helpful.