Dialogflow - Search and Conversation data store - updates on website url

[speralta]

Hello,

This is regarding Search and Conversation, chat with Dialogflow. I am using a Website URL as a data store. 

Does the data store automatically update it's index when there are changes and updates pushed on the website? If not, how do you handle using a website URL as a datastore when the site is constantly changing weekly?

[xavidop]

Hi, at the moment it need to be trigger manually: https://cloud.google.com/generative-ai-app-builder/docs/recrawl-websites

[harlanh]

Are you sure? When advanced website indexing is used, the docs say "We refresh the index of your actively used data stores every few weeks.". 

[xavidop]

mmm, but that is not too often, and it only works for Advanced NLU. 

[nimrah-waqar]

The website URL data store doesn't automatically update its index. You'll need to implement a scheduled job or webhook to regularly fetch and update data from the website to keep your Dialogflow agent's datastore current