Dataform Execution Error

[KMShaba]

I'm getting an error on Dataform after execution of the query where the failure status reason is "Not found; Gaia id not found for email ________"

What does the error mean and what's the solution to run the execution without any errors?

[ms4446 · Accepted solution]

The error message "Not found; Gaia id not found for email ________" in Google Cloud Dataform typically means that the service account used by Dataform to execute the query doesn't have the necessary permissions or isn't recognized by the system. This can happen due to a few reasons:

Potential Causes

1. Incorrect Service Account Configuration:

   • Missing or Incorrect Permissions: The service account might not have the required roles (e.g., BigQuery Data Editor, BigQuery Job User) to access and manipulate data in BigQuery or other resources.

   • Service Account Disabled or Deleted: The service account might be disabled or deleted in your Google Cloud project.

   • Workload Identity Federation Misconfiguration: If using Workload Identity Federation, ensure that it is correctly set up and that the appropriate service account is associated.

2. Project or Resource Access Issues:

   • Project Restrictions: There might be project-level restrictions or firewall rules preventing Dataform from accessing the necessary resources.

   • Resource Location Mismatch: Ensure that the Dataform project, the service account, and the resources (e.g., BigQuery datasets) are in the same region or have appropriate cross-region access enabled.

Solutions

1. Verify Service Account Permissions:

   • Go to the IAM & Admin section of your Google Cloud Console.

   • Locate the service account used by Dataform (check Dataform settings for this).

   • Check if the service account has the necessary roles to access the resources involved in your query (e.g., BigQuery). Grant the required roles if not.

2. Check Service Account Status:

   • Ensure that the service account is enabled in the IAM & Admin section.

   • If the service account has been deleted, recreate it with the same name and permissions.

3. Review Workload Identity Federation (if applicable):

   • Double-check the configuration to ensure it correctly links your Dataform environment to the appropriate service account.

4. Examine Project and Resource Access:

   • Check for any project-level restrictions or firewall rules that might be blocking Dataform's access.

   • Confirm that the Dataform project, service account, and the resources you're accessing are in the same region or have cross-region access enabled.

5. Dataform Troubleshooting:

   • Review the Dataform logs for more detailed error messages that might provide additional clues.

   • Check the Dataform documentation and community forums for troubleshooting tips specific to your issue.

[ms4446 · Accepted solution]

The error message "Not found; Gaia id not found for email ________" in Google Cloud Dataform typically means that the service account used by Dataform to execute the query doesn't have the necessary permissions or isn't recognized by the system. This can happen due to a few reasons:

Potential Causes

1. Incorrect Service Account Configuration:

   • Missing or Incorrect Permissions: The service account might not have the required roles (e.g., BigQuery Data Editor, BigQuery Job User) to access and manipulate data in BigQuery or other resources.

   • Service Account Disabled or Deleted: The service account might be disabled or deleted in your Google Cloud project.

   • Workload Identity Federation Misconfiguration: If using Workload Identity Federation, ensure that it is correctly set up and that the appropriate service account is associated.

2. Project or Resource Access Issues:

   • Project Restrictions: There might be project-level restrictions or firewall rules preventing Dataform from accessing the necessary resources.

   • Resource Location Mismatch: Ensure that the Dataform project, the service account, and the resources (e.g., BigQuery datasets) are in the same region or have appropriate cross-region access enabled.

Solutions

1. Verify Service Account Permissions:

   • Go to the IAM & Admin section of your Google Cloud Console.

   • Locate the service account used by Dataform (check Dataform settings for this).

   • Check if the service account has the necessary roles to access the resources involved in your query (e.g., BigQuery). Grant the required roles if not.

2. Check Service Account Status:

   • Ensure that the service account is enabled in the IAM & Admin section.

   • If the service account has been deleted, recreate it with the same name and permissions.

3. Review Workload Identity Federation (if applicable):

   • Double-check the configuration to ensure it correctly links your Dataform environment to the appropriate service account.

4. Examine Project and Resource Access:

   • Check for any project-level restrictions or firewall rules that might be blocking Dataform's access.

   • Confirm that the Dataform project, service account, and the resources you're accessing are in the same region or have cross-region access enabled.

5. Dataform Troubleshooting:

   • Review the Dataform logs for more detailed error messages that might provide additional clues.

   • Check the Dataform documentation and community forums for troubleshooting tips specific to your issue.

[KMShaba · Accepted solution]

Thank you for the quick response and possible solutions. 

[KMShaba]

Update: I was able to execute a table successful. Thanks again for the help