Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Unable to send SMTP Dag failure notifications

Hi,

 

 

I need to send failure notifications from composer DAGs. I have created an app password for gmail smtp. Following is the composer config override configuration. I created the password in secret manager.

 

smtp

smtp_port

587

 

smtp_timeout

30

 

smtp_mail_from

h******@gmail.com

 

smtp_host

smtp.gmail.com

 

smtp_starttls

True

 

smtp_ssl

False

 

smtp_retry_limit

5

 

smtp_user

h******@gmail.com

 

smtp_password_secret

smtp-password

email

email_backend

airflow.utils.email.send_email_smtp

 

 

 

 

I am getting the following error after running the DAG

 

Error

2022-02-01T15:23:27.361324593Z

airflow-worker (530, b'5.7.0 Authentication Required. Learn more at\n5.7.0 https://support.google.com/mail/?p=WantAuthError mv10sm3097322pjb.45 - gsmtp', 'h*******@gmail.com')

textPayload

(530, b'5.7.0 Authentication Required. Learn more at\n5.7.0 https://support.google.com/mail/?p=WantAuthError mv10sm3097322pjb.45 - gsmtp', 'h****@gmail.com')

 

However, when I run the following python code, it sends the email successfully

 

import smtplib, ssl

smtp_server = "smtp.gmail.com"

port = 25#587# For starttls

sender_email = "h***@gmail.com"

receiver_email = "h***@gmail.com"

password = "******"#input("Type your password and press enter: ")

message = "hello"

# Create a secure SSL context

context = ssl.create_default_context()

# Try to log in to server and send email

try:

    with smtplib.SMTP(smtp_server, port) as server:

        server = smtplib.SMTP(smtp_server,port)

        server.ehlo() # Can be omitted

        server.starttls(context=context) # Secure the connection

        server.ehlo() # Can be omitted

        server.login(sender_email, password)

        server.sendmail(sender_email, receiver_email, message)

    # TODO: Send email here

except Exception as e:

    # Print any error messages to stdout

    print(e)

finally:

    server.quit()

 

I tried all possible changes I could think of, deleting secret and editing the airflow.cfg directly to set the smtp config, changing values of config each, but nothing helped . Would appreciate any help, as I am completely stuck on this.

0 1 1,321
1 REPLY 1

Hello

If you could modify the entry smtp_password_cmd with the value "gcloud secrets versions access latest --secret=airflow-config-smtp-password" and that way the airflow.cfg will take the value for the variable smtp_password.

i.e    [smtp_password_cmd = gcloud secrets versions access latest --secret=airflow-config-smtp-password]