I have a Cloud Composer environment running with the 1.20.11 Composer version and 2.3.4 Airflow, but I have some difficulties with changing the airflow variables using the ci/cd pipeline in GitHub actions. I tried to use the command below and I am getting 'ERROR: (gcloud.composer.environments.run) PERMISSION_DENIED: The caller does not have permission' and I can't find the right permission to grant to the GitHub actions service account to run this.
gcloud composer environments run $COMPOSER_ENVIRONMENT --location $LOCATION variables import -- gcsfuse/variables.json
I don't know if we have some other options to update the variables to use in the scripts as you see below:
internal_project = Variable.get("bq_project")