Dataplex - Email alert on job failure - needs to happen once - incremental scope

[rajalukshmi]

Dear Team

Any inputs on the below would be highly appreciated?

we have set the below things in our scan

scope incremental

threshold - 0.99

notification report 

 job failure trigger - True

Scorethresholdtrigger -100%

in addition to other required parameters. we have configured the above parameters for 1000+ scans

We are seeing two alerts mails for each scans that fails. We are trying to understand how this works and how to restrict the mail alert to once per scans

[marckevin]

Hi @rajalukshmi,

Welcome to Google Cloud Community!

Here are some possible reasons and suggestions that you can try:

• Try examining your data quality scan results or Cloud Logging first, examine and review logs related to the two alert mails, this will give you more insight into the problem. Check the alert timestamps and identify if there are any retry attempts. You can also check for the exact failed rules.
• Make sure there are no other Cloud Logging-based alerts triggered by the same job failure, this might be the reason you're receiving two alert mails.
• Try recreating a simple scan with one rule and the same configuration for notifications. This will help determine if the two alert emails were caused by multiple rules (if applicable) in your scan, rather than a configuration issue.
• Review your DataQualityScan related setting. Try adjusting parameters or other settings based on your rules and requirements to see if you still receive two alert mails. Your setting might be triggering multiple job failures within your incremental scan, even though the intention is to trigger an alert only on job failure.

If the issue persists, I recommend reaching out to Google Cloud Support for further assistance, as they can provide insights into whether this behavior is specific to your project.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[rajalukshmi · Accepted solution]

It is the cloud scheduler triggering the scans twice causing multiple mails. Thanks for the input