Solved: Export Dialogflow CX agent data to BigQuery via Ag...

jasonjasonk · 01-23-2023 09:08 PM

I believe I have followed the instructions properly:

1) I have created a bigquery instance in the same project as my Dialogflow project

2) I created a dataset in that project

3) I used the SQL command to create the table, and the table is there with the correct columns

4) In Dialogflow CX agent settings I have ticket the box for BigQuery data export and I have supplied the name of the dataset and of the table, using the copy icon from big query (eg in the left side, clicking on the 3 dot menu to expose the sub-menu and then clicked "copy" for each the data set and table.

I have now had many chats take place both completed and abandoned sessions and it's been a few hours..still no data in big query (that I can see).. I'm totally new to BigQuery but I'm pretty sure that I have no data:

I click on the Table from the left menu, then click the "Details" tab and it says 0 rows. If It try to open the table via Google Sheets, the columns are there, no rows of data.

Does this feature need something else to work? The documentation on it basically says, set up the table, then enable the feature and fill in the fields...

In an old forum post, I found this medium article to just have the logs write to BQ... but seems like this feature would have the data maybe cleaner?

https://medium.com/google-cloud/dialogflow-cx-response-logging-e1b77d7a9fc6

Searched YouTube as well... and any Dialogflow to BQ tutorials are all for ES. Thanks for any help!

aaronscottb

Your BigQuery dataset must be in US multi-region and your Dialogflow CX agent must also be in US. This should already export your interactions/conversations successfully to your BigQuery.

View solution in original post

jasonjasonk

Aaron - thanks for such a quick reply.

Interesting to note, this morning there were still no results in the table.

While waiting to hear back, I ended up setting up the sync from the cloud logs to BigQuery as explained from this article https://loom.com/i/9d15cd8dfdc94ce88d6ef1f026fb599b

This worked, and now that I've seen your response, I went back and lo. there they are! The original table I created /inquired about has rows. As you said, it should already be exporting successfully, and it is.

The other possibility - I see that this table is partitioned daily.. did it need 24hours to start?

If not, it seems like there may be some other bug, which was resolved once I got some other sync working, and/or some other data into any big query table in that same project.

To be clear - I now have 2 tables in the same BigQuery instance, both now have my DialogflowCX data. 1 table is populating from the logs sync, the other is working via the integration I initially inquired about.

The 1st table (from my initial inquiry) seems to only have started populating data, once I had data populating into the 2nd table I'd created to receive the syncs from the logs.

Anyway.. one way or the other.. it works now. Thanks!