Solved: BigQuery Dataset IAM Condition issue

desmo

1. I have an sa with BigQuery JobUser and the following DataViewer IAM Conditon ( Type is Dataset) at project level
2. The condition is visible against datasets including the one shown (tables_QA) which matches the condition
3. When I authenticate as that sa
- bq ls tables_QA correctly returns list of tables whereas bq ls tables which also exists correctly returns an error
- however a query against a listed table in tables_QA throws the permissions error shown
- if DataViewer permission is added manually for the sa to tables_QA without Condition, the same query works

Any thoughts on why the IAM Conditon is not working?

1.

2.

3.

Alia_Sam

From what you’re describing and what’s visible in your condition builder setup, the issue is likely with how the condition is evaluated at query time, not at listing time.

Here's what’s likely happening:
Listing the dataset (bq ls) uses metadata access, which your Data Viewer role with condition seems to permit. Querying a table (bq query) involves access to table data, not just metadata, and this invokes a deeper evaluation of the IAM condition. Your condition matches on resource.name, which works for dataset-level IAM, but when querying a table, the resource context is the table, not the dataset.

To fix it, modify the IAM condition so that it applies based on the dataset name, regardless of the table name. You might need to explicitly use resource.name.startsWith (projects/.../datasets/...) if you're trying to scope access per dataset. Or, for testing, temporarily assign the roles/bigquery.dataViewer at the dataset level (not project) and without a condition to confirm this is indeed the root cause.

View solution in original post

Alia_Sam

From what you’re describing and what’s visible in your condition builder setup, the issue is likely with how the condition is evaluated at query time, not at listing time.

Here's what’s likely happening:
Listing the dataset (bq ls) uses metadata access, which your Data Viewer role with condition seems to permit. Querying a table (bq query) involves access to table data, not just metadata, and this invokes a deeper evaluation of the IAM condition. Your condition matches on resource.name, which works for dataset-level IAM, but when querying a table, the resource context is the table, not the dataset.

To fix it, modify the IAM condition so that it applies based on the dataset name, regardless of the table name. You might need to explicitly use resource.name.startsWith (projects/.../datasets/...) if you're trying to scope access per dataset. Or, for testing, temporarily assign the roles/bigquery.dataViewer at the dataset level (not project) and without a condition to confirm this is indeed the root cause.

desmo

Thanks for your response.
By using Resource Type = Dataset I didn't expect the condition to be applied to non-Datasets but with IAM conditions, it appears they are applied right down the hierarchy so both:
- Resource Type = Dataset and
- Ends with "_PROD"
get applied to objects in the Dataset container which is the problem
so startsWith is the only expression which works in a hierarchy

So IAM conditions are not quite as smart/flexible as I expected, still very useful though.