Issue with VPC Service Controls and Ingress Policy on Google Cloud

[reezz]

Description:
Hello,

In our organization, we have configured an Access Policy with a scope for a GCP folder.

Here are the details of our configuration:

1. Access Policy:

- Name: `accessPolicies/XXXXX`
- Scope: GCP Folder

2. **Access Level:**
- Name: `accessLevels/Access_Level_Restriction`
- Filter: Authorized public IPs

3. **VPC Service Perimeter in Dry Run mode:**
- Name: `servicePerimeters/vpc_perimeter`
- Resources to protect:
- Projects: All GCP projects in the GCP folder
- Networks: All VPCs of the protected GCP projects
- Restricted Services: All services
- VPC Accessible Services: N/A
- Access Levels: N/A
- Ingress Policy:
- From: Any Identity (Source: Access Level: `Access_Level_Restriction`)
- To: All projects, All services
- Egress Policy:
- From: Any Identity
- To: All projects, All services

We have enabled the VPC SC / WIF feature for our organization.

Issue Encountered:
When connected via Workforce Identity Federation (WIF) or Cloud Identity, I can access Compute Engine to list instances in the GCP project `project_id`. However, when filtering logs with `log_id("cloudaudit.googleapis.com/policy") AND severity="error" AND protoPayload.metadata.dryRun="true"`, I receive VPC Service Controls errors in Dry Run mode stating:

(Dry Run Mode) Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: XXXXXXX

**Error Details:**
```json
{
"protoPayload": {
"@type": "type.googleapis.com/google.cloud.audit.AuditLog",
"status": {
"code": 7,
"message": "(Dry Run Mode) Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: XXXXXXX",
"details": [
{
"@type": "type.googleapis.com/google.rpc.PreconditionFailure",
"violations": [
{
"type": "VPC_SERVICE_CONTROLS",
"description": "XXXXXXX"
}
]
}
]
},
"authenticationInfo": {
"principalEmail": "anonymized_email@domain.com"
},
"requestMetadata": {
"callerIp": "MY_IPV4",
"requestAttributes": {},
"destinationAttributes": {}
},
"serviceName": "compute.googleapis.com",
"methodName": "compute.beta.ProjectsService.Get",
"resourceName": "projects/XXXXXXXXX",
"metadata": {
"deviceState": "Unknown",
"ingressViolations": [
{
"servicePerimeter": "accessPolicies/XXXXX/servicePerimeters/vpc_perimeter",
"targetResource": "projects/XXXXXXXXX"
}
],
"intermediateServices": [
"cloudclient-pa.googleapis.com"
],
"securityPolicyInfo": {
"organizationId": "XXXXXXXXX",
"servicePerimeterName": "accessPolicies/XXXXX/servicePerimeters/vpc_perimeter"
},
"vpcServiceControlsUniqueId": "XXXXXXX",
"@type": "type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata",
"dryRun": true,
"resourceNames": [
"project_id"
],
"violationReason": "NO_MATCHING_ACCESS_LEVEL"
}
},
"insertId": "XXXXXXXXX",
"resource": {
"type": "audited_resource",
"labels": {
"method": "compute.beta.ProjectsService.Get",
"service": "compute.googleapis.com",
"project_id": "project_id"
}

Questions:

1. Why does this error appear even though the IP is authorized in the ingress policy?
2. Is it necessary to explicitly add the access level in the service perimeter despite the configuration via the ingress policy?

Thank you for your help!

[reezz · Accepted solution]

Hello,

Here's how we resolved the issue :

1. Explicitly Add Access Level in Service Perimeter:

   • Even though access levels can be configured using ingress policies, it is recommended to explicitly add the access level (Access_Level_Restriction) in the VPC Service Perimeter configuration. This can help ensure that the access level is recognized and enforced correctly.

2. Modify VPC Accessible Services and Restricted Services:

   • Ensure that the services you want to access (like compute.googleapis.com) are correctly listed in the VPC Accessible Services. This can be done by explicitly adding compute.googleapis.com to the VPC Accessible Services.

3. Ensure Organization-Level Access Policy:

   • Create an organization-level access policy without any specific scope. According to the troubleshooting documentation, if you don't have an organization-level access policy, you might experience unexpected issues with scoped access policies. Refer to Google Cloud VPC Service Controls Troubleshooting for more details.

Steps to Implement the Solution

1. Update Access Level in the Service Perimeter:

   gcloud access-context-manager perimeters update vpc_perimeter \ --policy=XXXXX \ --add-access-levels=accessLevels/Access_Level_Restriction

2. Add Compute Engine API to VPC Accessible Services:

   gcloud access-context-manager perimeters update vpc_perimeter \ --policy=XXXXX \ --add-vpc-allowed-services=compute.googleapis.com

3. Create Organization-Level Access Policy:

   gcloud access-context-manager policies create \ --organization=YOUR_ORG_ID --title=org_policy_no_scope

Verifying the Configuration

After making these changes, you can verify the configuration and test again:

1. Check Access Level Association:

   • Ensure that the Access_Level_Restriction is associated with the service perimeter by running:
     gcloud access-context-manager perimeters describe vpc_perimeter --policy=XXXXX

2. Test Access:

   • Try accessing the Compute Engine instances from an authorized IP and verify that there are no VPC Service Controls errors.
   • Ensure that unauthorized IPs still trigger the expected errors.

Conclusion

By explicitly adding the access level in the VPC Service Perimeter and ensuring the correct services are listed in the VPC Accessible Services, you should resolve the issue of unexpected VPC Service Controls errors. Additionally, creating an organization-level access policy without a specific scope can help mitigate unexpected issues with scoped policies.