When data of events_intraday table get migrated to the evets_dataset on BigQuery. As we have two tables events_intraday ( todays and yesterdays ) .
I have a cron job which extracting table data from bigQuery to GoogleCloudStorage Bucket using BQ Extract command.
Some time its throwing error that events_(date) table not found.
So I need to know exactly when events_intraday table push data to events table.
Is there any configuration where I can view/edit timestamp of migration.
Solved! Go to Solution.
Unfortunately, there is no configuration option to view or edit the timestamp of migration for the events_intraday table to the events table in BigQuery. This is a process managed by Google and is not exposed to users for modification.
The reason for this is that the migration of the events_intraday table to the events table is a complex process that requires careful coordination between Google Cloud and Firebase. If users were able to view or edit the timestamp of migration, this could potentially disrupt the migration process and lead to data loss.
Yes, there are a few workarounds you can consider:
Error Handling in Your Script: You could add error handling in your script to catch the "table not found" error. When this error is caught, you could have your script wait for a certain period of time and then retry the operation. This would give the data migration process more time to complete.
Use Both events_intraday_
and events_
Tables: If you need more real-time access to your data, you could modify your script to extract data from both the events_intraday_
and events_
tables. This would allow you to access the most recent data in the events_intraday_
table, as well as the historical data in the events_
table.
Use Google Cloud Functions: You could use Google Cloud Functions to trigger your data extraction process based on changes in your BigQuery dataset. For example, you could set up a Cloud Function to be triggered whenever a new table is added to your dataset. This would ensure that your data extraction process only runs after the data migration has occurred.