Vertex AI Agent - possible to fine-tune LLM or search tuning?

[Jakoblj]

Hi,

We have a vertex ai agent in production and need to improve performance further. Is it possible to use a fine-tuned LLM or search-tuning with a vertex ai agent (not vertex ai search app) to improve accuracy and handle company specific knowledge? And secondly is it possible to have more LLM calls in "one turn" to eg. search additional knowledge to include in the final answer or evaluate the answer before answering?

Thanks

[McMaco]

Hello Jakoblj,

Welcome to Google Cloud Community!

Fine-tune is under Generative AI on vertex AI, Parameter-efficient tuning and full fine-tuning are two approaches to customizing large models. Both methods have their advantages and implications in terms of model quality and resource efficiency.

Parameter efficient tuning

Parameter-efficient tuning, also called adapter tuning, enables efficient adaptation of large models to your specific tasks or domain. Parameter-efficient tuning updates a relatively small subset of the model's parameters during the tuning process.

To understand how Vertex AI supports adapter tuning and serving, you can find more details in the following whitepaper, Adaptation of Large Foundation Models.

Full fine-tuning

Full fine-tuning updates all parameters of the model, making it suitable for adapting the model to highly complex tasks, with the potential of achieving higher quality. However full fine tuning demands higher computational resources for both tuning and serving, leading to higher overall costs.

While Search Tuning was under Vertex AI Agent Builder, A tuned search model can give you better quality results than the base search model.

Search tuning is particularly valuable if you have industry-specific or company-specific queries that are less well addressed by general LLMs. It can be used to further train the search model.

Before you begin using search tuning, turn on Enterprise edition features for the app.

• To tune a search model with your own training data follow these steps.
• After tuning is complete, you can test it out by comparing the results of queries with the tuned model and the results of the same queries with the base model.
• After you have tested the tuned search and decided that you want to use it for all search queries, you can make it the default search model.

Yes it is possible to have more LLM calls in one turn, but we must keep in mind that the maximum number of LLM calls per playbook invocation in the same turn is 10.

I hope the above information is helpful.