Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

INFORMATION_SCHEMA.TABLES Monitoring last modified time

I'm monitoring table updates per project, so I'm using INFORMATION_SCHEMA.TABLES. But TABLES only have "creation time", don't have anymore LAST_MODIFIED_TIME like in INFORMATION_SCHEMA.SCHEMATA. I tried to monitor with TABLES_STORAGE_BY_PROJECT > STORAGE_LAST_MODIFIED_TIME but the result is different from the Last Modified Time from the Details when I see manually the table in Big Query.

What can I do to monitor the update of all the tables from a project? 

Solved Solved
1 7 22.6K
1 ACCEPTED SOLUTION

In Google Cloud BigQuery, the INFORMATION_SCHEMA.TABLES view does not include the LAST_MODIFIED_TIME column. However, you can use the __TABLES_SUMMARY__ meta-table to get the last modified time of a table.

Here's an example of how you can use it:

SELECT
table_id,
TIMESTAMP_MILLIS(last_modified_time) as last_modified_time
FROM
`your_project.your_dataset.__TABLES_SUMMARY__`

This will give you a list of all tables in the dataset along with their last modified time.

Please note that __TABLES_SUMMARY__ is a dataset-level meta-table, so you would need to run this query for each dataset in your project.

If you want to monitor the updates of all tables in a project, you might need to create a script or a scheduled query that periodically runs the above query for all datasets and then sends an alert or logs the result somewhere if a table has been updated.

Also, keep in mind that __TABLES_SUMMARY__ only provides information about native tables, not views or external tables. If you need to monitor updates to views or external tables, you would need a different approach, such as using Cloud Audit Logs to monitor changes to the underlying data sources.

View solution in original post

7 REPLIES 7