Announcements
This site is in read only until July 22 as we migrate to a new platform; refer to this community post for more details.

BigQuery Data Transfer job Failure Email Notifications

Former Community Member
Not applicable

We have automated a Bigquery data transfer job that loads data from the GCS bucket to the Bigquery table.

Everything is working fine. Currently, all failure email notifications are being set to the service account ID. but I am looking for a solution to get a failure email notification on our team email ID. Is there any way to set the DL-type email ID for the transfer job failure notification?

Please suggest a way to fix this issue or any alternative to it. What should we do so that all failure email notifications should be sent to our team email ID (DL), not our service account ID?

our transfer job configuration is as follows

 

 

transfer_config = bigquery_datatransfer_v1.TransferConfig(
        destination_dataset_id=dataset_id,
        display_name=f"{dataset_id}.{table_id}",
        data_source_id= config.get("data_source_id", "google_cloud_storage"),
        params={
            "destination_table_name_template": table_id,
            "write_disposition": "APPEND",
            "max_bad_records": config.get("max_bad_records"),
            "ignore_unknown_values": config.get("ignore_unknown_values"),
            "delete_source_files": config.get("delete_source_files",
            "file_format": config.get("file_format", "JSON"),
            "data_path_template": gcs_path
        },
        schedule=config.get("schedule", "every 20 minutes"),
        email_preferences=bigquery_datatransfer_v1.EmailPreferences(
            enable_failure_email=config.get("enable_failure_email", False)
        ),
        schedule_options=bigquery_datatransfer_v1.ScheduleOptions(
            start_time=start_timestamp,
        ),
        notification_pubsub_topic = None,
        owner_info = bigquery_datatransfer_v1.UserInfo(
            email = config.get("email", "")
        )
    )

 

 

 
0 1 757
1 REPLY 1

Thanks, helpful.