Mastering Cross-Project Service Account Impersonation in Google Cloud

[ameya-sap-cloud]

Working with Google Cloud often involves multiple projects and sophisticated permission models. A common, yet sometimes tricky, requirement is enabling a service account in one project to impersonate another service account in a different project. This is crucial for maintaining the principle of least privilege and streamlining access management.

However, developers frequently encounter “Permission denied” or “Invalid argument” errors when setting this up. Below is one such error which developers encounter while using ABAP SDK for Google Cloud.

This post will guide you through the complete process, leveraging a curl command from a Compute Engine VM to generate an access token for an impersonated service account across projects.

Note: This blog outlines a sample scenario for handling service account impersonation. It is not intended to be a guide for best practices. For best practice recommendations, please consult Google Cloud documentation.

The Challenge: Impersonating Across Project Boundaries (A Real-World Scenario)

Consider a common scenario faced by enterprises: a mission-critical SAP system with ABAP SDK deployed across multiple Google Cloud projects for disaster recovery.

• You have your primary SAP system and ABAP SDK deployed in main-site-proj (e.g., Mumbai region). This system interacts with various Google Cloud APIs (Google Bucket, Document AI, etc.) using a dedicated service account like abap-sdk-main@main-site-proj.iam.gserviceaccount.com.
• For disaster recovery, you have a standby SAP system in dr-site-proj (e.g., Delhi region).

During a DR switchover, the SAP system in dr-site-proj needs to take over, but it still requires access to the same Google API resources that are managed and provisioned in main-site-proj. The challenge arises when the SAP system (now running in dr-site-proj) tries to authenticate using its local VM’s service account to access resources tied to the service account in main-site-proj. This blog specifically addresses how to efficiently test this cross-project access without fully bringing up the entire DR SAP system.

So, for our example:

• A Source Project: dr-site-proj
• A Source VM’s Service Account: 942160574421-compute@developer.gserviceaccount.com (from dr-site-proj, running your test code)
• A Target Project: main-site-proj
• A Target Service Account: abap-sdk-main@main-site-proj.iam.gserviceaccount.com (the SAP system’s service account in the primary project)

The goal is to get a short-lived access token for abap-sdk-main@main-site-proj.iam.gserviceaccount.com while running code on a VM associated with 942160574421-compute@developer.gserviceaccount.com. Out-of-the-box, this won’t work, leading to frustrating errors like:

• Request had insufficient authentication scope
• Permission ‘iam.serviceAccounts.getAccessToken’ denied on resource (or it may not exist).
• Request contains an invalid argument.

Visualizing the Problem

This diagram illustrates the cross-project service account impersonation challenge:

The diagram shows that the VM in dr-site-proj (and its service account) needs a way to act as the abap-sdk-main service account in main-site-proj to access resources like Google Cloud Storage or Document AI which are configured within the main-site-proj context.

Setup Overview Table

To clarify the roles and identities in this cross-project setup, refer to the table below:

Prerequisites

Before we dive in, ensure you have:

1. A Compute Engine VM running in your dr-site-proj project.
2. The abap-sdk-main@main-site-proj.iam.gserviceaccount.com service account already created in your main-site-proj.
3. A request.json file on your VM with the following content:

{
 "scope": [
 "https://www.googleapis.com/auth/cloud-platform"
 ],
 "lifetime": "3600s"
}

(Note: An empty {} often results in “Invalid Argument” for this specific API.)

Step 1: Configure VM Access Scopes (on dr-site-proj VM)

The VM where your curl command runs needs to have sufficient scopes to request the initial token from its own metadata server, which will then be used to call the IAM Credentials API.

1. Switch to the dr-site-proj project in your Google Cloud Console.
2. Navigate to Compute Engine > VM instances.
3. Select your VM instance.
4. Stop the VM instance (scopes cannot be changed on a running VM).
5. Click the “Edit” button.
6. Scroll down to the “Service Account” section.
7. Under “Access scopes”, select “Allow full access to all Cloud APIs”. Security Note: While this is the easiest for testing, in production, consider the more granular https://www.googleapis.com/auth/iam scope if it meets your needs.
8. Click “Save”.
9. Start your VM instance again.

Step 2: Grant IAM Permission for Impersonation (Cross-Project)

This is the most critical step to overcome the “Permission denied” error. Your dr-site-proj VM’s service account needs explicit permission to generate tokens for the abap-sdk-main service account. This permission must be granted directly on the target service account.

1. Switch to the main-site-proj project in your Google Cloud Console.
2. Navigate to IAM & Admin > Service Accounts.
3. Locate and click on the target service account: abap-sdk-main@main-site-proj.iam.gserviceaccount.com.
4. Click on the “Principals with access” tab. This tab explicitly lists who can access or act as this specific service account.
5. Click the “Grant Access” button.
6. In the “New principals” field, carefully type or paste the full email address of your VM’s service account from the dr-site-proj project: 942160574421-compute@developer.gserviceaccount.com
7. In the “Select a role” dropdown, search for and select “Service Account Token Creator” (roles/iam.serviceAccountTokenCreator).
8. Click “Save”.

Step 3: Execute the curl Command

Now, with the correct IAM permissions and VM scopes in place, you can execute the curl command from your VM in dr-site-proj.

SSH into your VM instance in the dr-site-proj project. Ensure your request.json file (as defined in Prerequisites) is in the same directory. Please ensure to update the service account and make it relevant to your project.

# Part 1: Get the access token for the current VM's service account
# (942160574421-compute@developer.gserviceaccount.com)
ACCESS_TOKEN=$(curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" -H "Metadata-Flavor: Google" | sed -n 's/.*"access_token":"\([^"]*\)".*/\1/p') && \
\
# Part 2: Use that token to call the IAM Credentials API to generate a token
# for the target service account (abap-sdk-main@main-site-proj.iam.gserviceaccount.com)
curl -X POST \
     -H "Authorization: Bearer ${ACCESS_TOKEN}" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/abap-sdk-main@main-site-proj.iam.gserviceaccount.com:generateAccessToken"

Here is an example error response when IAM permission are not granted.

Once granted as describe in step2, the script will generate the required token and validation error will be resolved.

Conclusion

Setting up cross-project service account impersonation requires careful attention to both VM access scopes and explicit IAM role grants on the target service account. By following these detailed steps, you can successfully implement secure and efficient cross-project communication in your Google Cloud environment, addressing challenges like testing DR scenarios without full system bring-up and avoiding common “Permission denied” and “Invalid argument” errors.