Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Dataform error "Admin requires that you specify encryption key for queries"

Good day, today a policy was applied to make mandatory the use of KMS keys for everything including queries. I have succesfully managed to apply kms key to my dataflow jobs queries in apache beam thanks to the .withKmsKey(String kmsKey) function, but for dataform I have not been able to solve this error, let me share all what I have tried first:

Tried without success:

1) Creating from 0 the dataform repo using the same kms key that is used at the bigquery datasets that sqlx script is trying to query.

2) Tried to give the Service account used at my dataform workflows the encrypt/decrypt role to the key that is used at the bigquery dataset we are trying to query.

3) Finally in the documentation I found the flag to setup the kms_key_name at the sqlx scrypt config block, but this did also not work, here is the documentation I found about this:

* dataform core general doc: https://cloud.google.com/dataform/docs/reference/dataform-core-reference then in the IBigQueryOptions:

https://cloud.google.com/dataform/docs/reference/dataform-core-reference#bigquery, then inside this there is the additionalOptions parameter that contains the "kms_key_name" option: 
 
But this also did not work, I always get this error message:
 
"Your administrator requires that you specify an encryption key for queries in project project-id. See https://cloud.google.com/bigquery/docs/customer-managed-encryption#services_constraint for more info."
 
So right now I am not sure what else should we try? Please help
0 3 914
3 REPLIES 3