How to build an AI agent to look up both the datastore and internet?

[wih16]

I have been using the Goolgle Cloud Console to create my own AI agents, and so far it is successful to connect and use datastore (bucket) I created with my own documents. However I can't find the Settings option to enable access for AI Agent to use public internet as (backup) source of data too.

I asked Gemini, and its directions look to be different from what is available for me in the console. Has anybody built an agent to use own datastore (perhaps as primary source) and also use public internet to generate responses? Could you share your steps and knowledge?

This is what I asked Gemini: how to set up the AI agent in Vertex AI to be able to access both datastore information and public internet.

This is what Gemini responded, but somehow I can't follow the steps to find the option in the console: 

In Vertex AI, you can set up your AI agent to access both datastore information and the public internet using the following approach:

1. Datastore Integration:

• Create a Datastore: Set up a datastore within Vertex AI to hold your structured or unstructured data. You can choose from several data sources like website URLs, BigQuery, Cloud Storage, Google Drive, or Cloud SQL.
• Configure Datastore Access: In your agent's settings, under the "Data Store" section, connect your agent to the relevant datastore. You might need to define access permissions and specify how the agent should query the datastore.

2. Internet Access (Search Integration):

• Enable Search: In your agent's settings, under the "Generative" section, enable the "Search" feature. This will allow the agent to access Google Search to retrieve information from the public internet.
• Configure Search Parameters: Customize search parameters like the number of results to fetch and how to prioritize information from search versus the datastore.

3. Agent Logic:

• Design Prompts and Flows: Develop prompts and conversational flows that guide the agent to use both the datastore and internet search appropriately. For example:
  • If the user asks about a specific product, the agent can first check the product catalog in the datastore.
  • If the datastore doesn't contain the answer, the agent can then use internet search to find relevant information.
• Handle Responses: Implement logic in your agent to process and combine responses from both the datastore and internet search to provide comprehensive and accurate answers to the user.

Additional Considerations:

• Security: Be mindful of security when accessing external data sources like the internet. Consider filtering or validating the information retrieved from the web.
• Cost: Accessing external data sources might incur additional costs, so monitor your usage and adjust settings accordingly.

[nceniza]

According to Agent builder documentation here. An application has a many to one relationship with a data store. An Application can only connect to one data store while an data store can connect to multiple apps :

• A media app has a many-to-one relationship with its data store. An app can only connect to one data store, whereas a given data store can be connected to several apps. For example, a media search app and a media recommendations app can share a data store.

---

[ammar_hanif]

Well as per my experience and the provided link Relationship between app and datastore you can connect multiple datastore to a single app, which is known as blended  search. It can only be while creating the app and not after anytime.

However, there are certain limitations to it, which you can find in the link provided and to see if that suffice your purpose.