Using Workload identity for pulling images from private Artifact Registry

[cloudlearner]

Hi,

I have an image in Artifact Registry and have setup Workoad identity federation for Kubernetes by following instructions from this document. My kubernetes is an K3S light weight install. I am able to perform various activities like sending logs to GCP etc.

I now want to pull an image from Artifact Registry, I gave required permission to Service account but I am not able to pull the image. Is it possible to pull the image with Workload Identity ? If so, is there any documentation that I can pull ? I can only see one possibility of authenticating docker with gcp in docker and upload docker credential as imagePullSecret but I don't want to use static secrets here but use WIF. Any help in this regard ?

Thank you.

[RonEtch]

Hi @cloudlearner 

Welcome to Google Cloud Community!

Since it is not a GKE cluster, I think you may need imagePullSecrets on your deployment YAML.

Considering that you you've assigned proper roles/permission on your service account to access Artifact Registry, you will need to create serviceAccount and secret inside your cluster.

You can do the following summary steps to create serviceAccount and access the registry:

• Create a service account key on the console for the service account that will be used.
• Create a kubernetes service account inside the cluster and

kubectl create serviceaccount KSA_NAME \
--namespace NAMESPACE \
iam.gke.io/gcp-service-account=GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com

KSA_NAME: the name of your new Kubernetes service account.
GSA_PROJECT: the project ID of the Google Cloud project of your IAM service account.
GSA_NAME: the name of your IAM service account.

• Create secret inside of your cluster as follows:

kubectl create secret docker-registry gcr-json-key \
--docker-server=us.gcr.io \
--docker-username=_json_key \
--docker-password="$(cat <path-to-json-key>)"

You can check the following guide to know more on how to access ACR using a service account.

You can also check this article for added information on "how to" access your ACR using service account

I hope this information is helpful.

If you need further assistance, you can always file a ticket on our support team.

[cloudlearner]

Thanks for the reply .

I see that the above solution requires service account key which never expires and I need to have a. private part of the service account key in imagePullSecret. I have opted for Workload identity federation to avoid using private key which is unsecured.

Is there any way I can leverage Workload identity federation to access GAR ?

Thank you.

[RonEtch]

Hi @cloudlearner 

Have you tried also following this guide in your case ?

[nimrah-waqar]

1. Check service account permissions (roles/artifactregistry.reader).
2. Verify Workload Identity mapping between Kubernetes and GCP.
3. Test image pulling with kubectl.
4. Review relevant documentation.
5. Consider alternative approaches if necessary.
6. Seek community support if issues persist.

[paulleroy]

I have partially solved this, a little more tinkering is required. Basically if you run this it will update your pull secret to allow a pull from GAR, you'd need to run a workload identity federated container to periodically update the pull secret using this script. I have tested this without the automation and it works, so I just need to package this into a script and create a 3600s cronjob

```bash

EMAIL=<sa email addy>

echo "{\"auths\": {\"europe-west1-docker.pkg.dev\": {\"auth\": \"$(echo oauth2accesstoken:$(gcloud auth print-access-token --impersonate-service-account $EMAIL) | base64 -w0)\"}}}" > .dockerconfigjson_wif

k delete secret pkg-json-key
k create secret docker-registry pkg-json-key --from-file=.dockerconfigjson=.dockerconfigjson_wif

```

Then your 

pod yaml just needs to reference the imagepullsecret

[paulleroy]

Here is a sample script to run in a WIF enabled gcloud pod on your cluster to update the pull keys, you'd need to set the appropriate roles and rolebindings and set the cronjob to run before the end of every token expiry

#!/bin/bash

set -ex

export PATH="/root/google-cloud-sdk/bin/:$PATH"
gcloud auth login --cred-file=$GOOGLE_APPLICATION_CREDENTIALS
NEWPULLTOKEN=$(echo "{\"auths\": {\"europe-west1-docker.pkg.dev\": {\"auth\": \"$(echo oauth2accesstoken:$(gcloud auth print-access-token ) | base64 -w0)\"}}}" | base64 -w0)
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

cat > new-secret.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
name: pkg-json-key1
data:
.dockerconfigjson: "$NEWPULLTOKEN"
type: kubernetes.io/dockerconfigjson
EOF

curl -s --header "Authorization: Bearer $TOKEN" --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
--request PUT \
--header "Content-Type: application/yaml" \
--data-binary @New-secret.yaml \
https://kubernetes/api/v1/namespaces/default/secrets/pkg-json-key1

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dockerpullsecrets-updater
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["secrets"]
  resourceNames: ["pkg-json-key1"]
  verbs: ["*"]