Vertex AI: NLP Search on Application Logs

[priyance]

Hello community,

I am new to Vertex AI currently exploring different usecases of it.

Currently, I am trying to build Vertex Search App on our internal software logs to make it easy for our engineers to search through logs using Natural Language.

I am trying to tune the base model as I want to feed in domain specific knowledge to it. What are some techniques which I can use?

I have already tried boost/bury and filtering based on search queries. I am trying to do something to tune base model with domain specific queries and knowledge.

Any help will be appreciated, thanks in advance.

[a_aleinikov]

Hello @priyance ,

To tune a base model for domain-specific search, you can use techniques like prompt engineering, embedding your domain-specific data using Vertex AI Matching Engine, or fine-tuning a foundation model if available.
You might also consider creating a custom retriever using embeddings for better relevance.

[dawnberdan]

Hi @priyance,

Welcome to Google Cloud Community!

Fine-tuning a model to better handle domain-specific queries and knowledge is an effective way to enhance search result relevance. Since you've already experimented with boost/bury and filtering, Here are a few additional techniques you could explore:

1. Embedding-based Retrieval: For structured datasets, consider fine-tuning a domain-specific embedding model. This approach enables the creation of a semantic search model that leverages embeddings to better understand the relationships between logs and queries.

2. Retrieval-Augmented Generation (RAG): Use a hybrid method where you retrieve the most relevant log snippets and then employ a generative model (like Vertex AI's language models) to produce more detailed and cohesive answers.

3. Custom Tuning with Vertex AI: Vertex AI supports fine-tuning models using domain-specific datasets. By creating a dataset with labeled query-log pairs, you can fine-tune a foundational model to improve its ability to rank log entries based on domain knowledge.

Here are some documentation that might be useful for fine-tuning and search applications:

• Vertex AI Search
• Search Tuning in Vertex AI
  

Was this helpful? If so, please mark this as "Solution." If you need further assistance, feel free to reply within 2 business days, and I'll be happy to help.