Workload Identity Federation : AccessDeniedException: 403 Permission 'iam.serviceAccounts.getAccessT

[Loic76]

Hi everyone,

I’m working on a new GitLab CI pipeline with GCP. I’ve created a service account, but due to security restrictions, I cannot create an SSH key. Instead, I’m using Workload Identity Federation. I’ve set up a Pool and added my provider.

Here’s what my CI configuration looks like:

deploy_to_gae:
  stage: deploy
  id_tokens:
    ID_TOKEN:
      aud: "//iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../providers/..."
    SIGSTORE_ID_TOKEN:
      aud: "sigstore"
  tags:
    - runner1
  script:
    - echo ${ID_TOKEN} > .ci_jwt_v2
    - gcloud iam workload-identity-pools create-cred-config projects/.../locations/global/workloadIdentityPools/.../providers/... --service-account="...@...iam.gserviceaccount.com" --output-file=.gcp_credentials.json --credential-source-file=.ci_jwt_v2

    - gcloud auth login --cred-file=`pwd`/.gcp_credentials.json
    - cat .gcp_credentials.json
    - gcloud config set project $GCP_PROJECT_ID
    - gcloud app deploy --quiet
  only:
    - main

However, I’m encountering this error in the logs:

$ echo ${ID_TOKEN} > .ci_jwt_v2
$ gcloud iam workload-identity-pools create-cred-config projects/.../locations/global/workloadIdentityPools/.../providers/... --service-account="...@....iam.gserviceaccount.com" --output-file=.gcp_credentials.json --credential-source-file=.ci_jwt_v2
Created credential configuration file [.gcp_credentials.json].
$ gcloud auth login --cred-file=`pwd`/.gcp_credentials.json
Authenticated with external account credentials for: [...@....iam.gserviceaccount.com].
Your current project is [None].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID
$ cat .gcp_credentials.json
{
  "universe_domain": "googleapis.com",
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../providers/...",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": ".ci_jwt_v2"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/...@....iam.gserviceaccount.com:generateAccessToken"
}$ gcloud config set project $GCP_PROJECT_ID
ERROR: (gcloud.config.set) There was a problem refreshing your current auth tokens: ('Unable to acquire impersonated credentials', '{\n  "error": {\n    "code": 403,\n    "message": "Permission \'iam.serviceAccounts.getAccessToken\' denied on resource (or it may not exist).",\n    "status": "PERMISSION_DENIED",\n    "details": [\n      {\n        "@type": "type.googleapis.com/google.rpc.ErrorInfo",\n        "reason": "IAM_PERMISSION_DENIED",\n        "domain": "iam.googleapis.com",\n        "metadata": {\n          "permission": "iam.serviceAccounts.getAccessToken"\n        }\n      }\n    ]\n  }\n}\n')

I’ve already granted several permissions (I’ll remove unnecessary roles later), but the error persists. Here are the permissions I’ve applied:

bindings:
- members:
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/appengine.serviceAgent
- members:
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/appengineflex.serviceAgent
- members:
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/iam.serviceAccountAdmin
- members:
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/iam.serviceAccountOpenIdTokenCreator
- members:
  - principal://iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../subject/...
  - principalSet://iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../attribute.repository/clients/...
  - principalSet://iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../attribute.repository/...
  - principalSet://iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../attribute.repository/...
  - serviceAccount:...@....iam.gserviceaccount.com
  - user:...@...
  role: roles/iam.serviceAccountTokenCreator
- members:
  - principalSet://iam.googleapis.com/projects/.../locations/global/workloadIdentityPools/.../attribute.repository/clients/...
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/iam.serviceAccountUser
- members:
  - serviceAccount:...@....iam.gserviceaccount.com
  role: roles/iam.workloadIdentityUser
etag: BwYiTdmWSfM=
version: 1

If anyone has any ideas or solutions, your help would be greatly appreciated!

[mcbsalceda]

Hi @Loic76,

Welcome to Google Cloud Community!

It looks like you might be missing the IAM Workforce Pool Admin role (roles/iam.workforcePoolAdmin). According to the troubleshooting guide, this error usually pops up when the user trying to set up Workforce Identity Federation doesn't have that specific role. Adding it should help resolve the issue.

I hope the above information is helpful.

[Loic76]

Hi @mcbsalceda, 

Thank you for your input! I appreciate the information you shared regarding Workforce Identity Federation. However, I’ve realized that my use case involves Workload Identity Federation.

I tried adding the roles/iam.workloadIdentityUser role (which is the only role for Work  Federation available), but I am still encountering a 403 Permission Denied error.

[mcbsalceda]

Hi @Loic, 

Could you check if you’ve set $GCP_PROJECT_ID on line 16 in your CI configuration to your project ID? Also, make sure the credential file path is correct. If updating these doesn’t resolve the issue, this documentation might help you double-check your setup.

[Loic76]

Hi @mcbsalceda 

I do have the correct project ID (this is not the project number), and as proof, my graph in Workload Identity Federation changes with each of my attempts.

I reviewed the provided documentation, and generateIdToken seems to be for OAuth 2 authentications with some humain validation.

I’m wondering about the attributes I mapped in the pool provider; maybe I’m not sending the right information with my CI? Or is my audience incorrect?

Thank you again for your help.
Loic.