Retrieval tool not called by the GenerativeModel

[lgubert]

Hey everyone!

I'm trying to implement a RAG agent following the available documentation on VertexAI and Gemini, and I'm not getting the desired results when using the GenerativeModel versus querying the corpus directly. Here's a working example:

vertexai.init(credentials=credentials, project=PROJECT_NAME, location='us-central1')

CORPUS_NAME = "projects/..../locations/us-central1/ragCorpora/...."

rag_tool = Tool.from_retrieval(
    retrieval=rag.Retrieval(
        source=rag.VertexRagStore(rag_corpora=[CORPUS_NAME], similarity_top_k=3)
    )
)

# Direct context retrieval
response = rag.retrieval_query(
    rag_corpora=[CORPUS_NAME],
    text="<Question about the document>",
    similarity_top_k=10,  # Optional
)
print(response)  # This gives a satisfactory response based on the document's content.

model = GenerativeModel(
    model_name="gemini-1.5-flash",
    tools=[rag_tool]
)

answer_0 = model.generate_content("<Question about the document>")
print(answer_0.text)  # This answer does not seem to acknowledge the existence of the document

chat = model.start_chat()
answer_1 = chat.send_message("<Question about the document>")
print(answer_1.text)  # This answer also does not seem to acknowledge the existence of the document

The retrieval_query answer is pretty good, but in the last two cases when using the model, I get responses like "I need the search results to answer your question. Please provide them." or "Please provide me with the search results so I can answer your question."

The corpus I'm using has a single PDF file in it.

Has anyone noticed this behavior as well or understands if something in my code is incorrect?

[cassandramae · Accepted solution]

Hi @lgubert,

Welcome to Google Cloud Community!

Is this only happening to gemini-1.5-flash? Have you considered changing the model to gemini-1.5-flash-001 or other available models? This will help us isolate in case this is a model issue. You may also try running this sample code or quickstart as a baseline for your implementation and troubleshooting.

I hope this helps.

[lgubert]

@cassandramae turns out the default value for vector_distance_threshold (0.3) was too low for my use case, so after setting it to 0.5 like the example, it worked as intended. Thanks!

[sailochana]

HI ,
I actually got blank ,I uploaded files and even got ragfile corpus id ,but while query with the data using rag.retrieval_query() ,i got just blank 
can anyone help me on this