Solved: Dataplex attribute store on entities from bigquery - Page 2

Victor07hl · 04-16-2024 10:30 AM

Hi all,

I'm trying to attach an Attribute Store to a table in BigQuery that already existed before I created a Dataplex asset mapping the entire dataset. While I can query the table successfully through Dataplex, I encounter a warning message when attempting to attach an Attribute Store: "Updating IAM policy via Dataplex not supported for the resource associated with entity of name."

this means the Attribute Store isn't working correctly. Are there specific requirements for attaching an Attribute Store to a pre-existing BigQuery table within a Dataplex lake's zone?

ms4446

Hi @Victor07hl ,

The warning message, "Updating IAM policy via Dataplex not supported for the resource associated with entity of name" usually means Dataplex needs explicit permission to manage Attribute Stores for tables it didn't create.

Here's how to fix it:

1. Grant IAM Roles Manually:

In the Google Cloud Console, go to BigQuery -> [Your Dataset] -> Permissions.
Add the Dataplex service account (typically service-PROJECT_NUMBER@gcp-sa-dataplex.iam.gserviceaccount.com).
Grant these roles:
- BigQuery Data Owner: For managing table data.
- Dataplex Metadata Admin: Essential for metadata operations.

2. Use Terraform or gcloud:

If you're using infrastructure as code, add the roles to your scripts. Here's a gcloud example:

 

gcloud projects add-iam-policy-binding [PROJECT-ID] \
  --member="serviceAccount:service-[PROJECT_NUMBER]@gcp-sa-dataplex.iam.gserviceaccount.com" \
  --role="roles/bigquery.dataOwner"

Important Considerations:

Project vs. Dataset-Level Roles: Dataset-level is more secure, but project-level might be simpler for multiple tables.
Asset Type: Make sure it's set as "BigQuery table" in Dataplex.

After updating permissions, try attaching the Attribute Store again. It should work smoothly now!

View solution in original post

ms4446

Hi @Victor07hl ,

Yes, you are right – Dataplex Metadata Admin isn't a pre-defined role in BigQuery IAM. It's actually a custom role you'll need to create within Dataplex itself. Here's how:

1. Create the Custom Role:

In the Google Cloud Console, go to Dataplex -> IAM & Admin -> Roles.
Click Create Role.
Give it a name like "Dataplex Metadata Admin" and a description.
Under Permissions, add the following:
- dataplex.entities.updateMetadata
- dataplex.entities.getMetadata
- dataplex.entities.list (if you want to allow listing entities)
Click Create.

2. Grant the Custom Role:

Go back to BigQuery -> [Your Dataset] -> Permissions.
Find the Dataplex service account you added earlier.
Click Grant Access.
In the Select a role dropdown, type the name of your custom role (e.g., "Dataplex Metadata Admin") and select it.
Click Save.

Cross-Project Setup:

Yes, having the BigQuery dataset in a separate project can affect things. Make sure the Dataplex service account from your Dataplex project also has the necessary permissions in the BigQuery project. You might need to grant it the BigQuery Data Viewer role in the BigQuery project as well.

After these steps:

Try attaching the Attribute Store again. If everything is set up correctly, it should work this time!

View solution in original post