Dataplex Data Quality Scan Rules. Unable to Filter Active/Suspended Rules

[svetshum]

When I run a Data Quality scan with multiple rules, the BigQuery result table includes entries for all rules, regardless of whether they are active/enabled or not. And the result table in BQ does not contain any related attribute (like enabled, rule_status) that would allow to filter the results. 

Are there any way to configure the scan so that only active/enabled rules are included in the results. Or how the "Actions" option is supposed to function in this context ?

[jangemmar]

Hi @svetshum,

Welcome to Google Cloud Community!

Data Quality scans are designed for comprehensive analysis, meaning these scans provide a full picture of your data quality, so directly filtering results based on the active/enabled status of the rules is not available at the moment. In other words, BigQuery result table includes data for all rules, regardless of their status.

However, the best workaround that you can do is to use BigQuery to filter the results based on the rule_name column, identifying active rules based on your configuration. Use a WHERE clause in your query to filter the results based on the rule names you have specifically enabled.

Feel free to utilize these sample queries to set per rule alerts in Cloud Logging.

Also, monitor these Dataplex release notes to keep you updated for new features, bug fixes, known issues, and deprecated functionality.

I hope the above information is helpful.