Dialogflow Source Documents/Customization

[jordanshans]

I am building a DialogFlow Agent where my datastore is website content.

1. I am trying to find where I can adjust the number of sources that Vertex AI Search queries and are used in the prompt to the LLM. I know now there are 10 and I want to adjust that.

2. I also can't find my current schema to possibly use meta tags as filters, but have used google generated fields "datePublished" and "dateModified" for query operators. 

3. Is there a way so that the datastore response does not always output a link? using a threshold?

4. My main issue is that the agent outputs irrelevant links that are in the sources just because it was the first link. How do I control this? 

Thank you in advance 

[cassandramae]

Hi @jordanshans,

Welcome to Google Cloud Community!

Please see my answers in line with your questions below:

1. If you’re referring to summary result count for top results, it seems that it’s not possible to adjust this to more than 10 as the maximum value is 10. However, if you need to show more than 10 search results, you may consider using this method.

2. Since you're using website content, note that viewing the schema for a website is supported only over the REST API as mentioned here. With this, you may use the schemas.get API method below to get your schema definition.

curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/schemas/default_schema"

3. You may consider using search configuration to have better control over the agent behavior and improve the quality of the answers.

4. It could be that the data is not yet fully indexed. Note that it can take up to 4 hours for new data store web content to be indexed and properly handled by your agent.

Hope this helps.

[jordanshans]

Thank you for your response. I’m still trying other ways to enhance it, and I think I understand the problem better. 

Im using the link output DataStore response in the dialog flow console. Often the link that is output in this fulfillment isn’t necessarily the ideal link from the sources. My idea was to use a webhook and process the datastore agent output (where it gives each answerParts with its supportingIndices) and based on logic from my use case, choose the most relevant source and output it’s link. 

I cannot manage to extract the full datastore output in order to process it. How can I do so? 

thank you in advance