Seeking Help to Resolve Access Issues with Google Cloud Load Balancer and CDN

[Yash_A]

Hey GCP Community,

I'm feeling a bit stuck and would really appreciate your help. I've set up a Google Cloud Global External Application Load Balancer with a backend bucket and enabled CDN. I also added an SSL certificate to ensure all traffic is securely routed to HTTPS.

However, when I try to access the storage files through the CDN, I keep getting an AccessDenied Error (403). I want to ensure that only the load balancer can access the bucket to fill the CDN cache, but I'm not sure how to set it up correctly.

I found two possible solutions online:

1. Make the Storage Bucket public: I don't want to do this, as I want the bucket's access restricted to just the load balancer.
2. Grant the load balancer's service account access to the bucket: The problem is, I can't find any service account related to the load balancer.

Has anyone encountered a similar issue or can offer guidance? I want to resolve this without making the bucket public or giving access to anyone else. Any advice or steps would be greatly appreciated!

Thank you so much in advance!

[yrhsk]

Search for the compute SA (something like service-PROJECT_NUMBER@compute-system.iam.gserviceaccount.com), and assign the `Storage Object Viewer` role to it in the Cloud Storage -> Bucket -> Permissions. It should resolve the issue

[Yash_A]

I have granted the storage object viewer role to compute SA, but still the CDN is giving AccessDenied error.

[reachnavya]

Hello Yash, 

You have two options - 

1. Use signed url - https://cloud.google.com/cdn/docs/authenticate-content#how_signed_urls_work. This will require the request made the client to be signed too. 

2. Use private origin authentication - https://cloud.google.com/cdn/docs/configure-private-origin-authentication . You can refer to this medium post on detailed instruction on how you can use this feature to access private GCS buckets - https://medium.com/@thetechbytes/private-gcs-bucket-access-through-google-cloud-cdn-430d940ebad9

[Yash_A]

I tried to follow the medium's' article. In Step 7d, I'm facing issues while uploading the yaml file to backend service

For testing, I tried to upload the same file, which is created by GCP, without any editing, but that is also failing

The code that I followed on Windows PowerShell:

gcloud beta compute backend-services describe cdn-gcp-neg-lb-service --global > cdn-private-origin.yaml; gcloud compute backend-services import cdn-gcp-neg-lb-service --source cdn-private-origin.yaml --global

The Error that I'm getting:

ERROR: gcloud crashed (UnicodeDecodeError): 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics

[yrhsk]

As far as I understand, it's the encoding issue. Try running PowerShell commands to reencode  ```

Get-Content cdn-private-origin.yaml -Encoding Byte | Set-Content cdn-private-origin-utf8.yaml -Encoding utf8

Then try to run `gcloud` with the updated file:

gcloud compute backend-services import cdn-gcp-neg-lb-service --source cdn-private-origin-utf8.yaml --global

[reachnavya]

You would have to make the changes mentioned in 7c before you can reuse the config. Below is a sample complete config that you can use as a reference. Try adding the optional accessKeyVersion under awsV4Authentication in case the below doesnt work. 

Sample config - 

affinityCookieTtlSec: 0

backends:

- balancingMode: UTILIZATION

capacityScaler: 1.0

group: https://www.googleapis.com/compute/v1/projects/sample/global/networkEndpointGroups/private-gcs

cdnPolicy:

cacheKeyPolicy:

includeHost: true

includeProtocol: true

includeQueryString: true

cacheMode: CACHE_ALL_STATIC

clientTtl: 3600

defaultTtl: 3600

maxTtl: 86400

negativeCaching: false

requestCoalescing: true

serveWhileStale: 0

compressionMode: DISABLED

connectionDraining:

drainingTimeoutSec: 0

creationTimestamp: '2024-06-25T16:12:26.040-07:00'

customRequestHeaders:

- host:private-bucket.storage.googleapis.com

description: ''

enableCDN: true

kind: compute#backendService

loadBalancingScheme: EXTERNAL_MANAGED

localityLbPolicy: ROUND_ROBIN

logConfig:

enable: true

optionalMode: EXCLUDE_ALL_OPTIONAL

sampleRate: 1.0

name: private-gcs

port: 80

portName: http

protocol: HTTPS

securitySettings:

awsV4Authentication:

accessKeyId: <yourKeyID>

accessKey:<yourAccessKey>

originRegion: us-east-2

selfLink: https://www.googleapis.com/compute/v1/projects/sample/global/backendServices/my-lb-bs

sessionAffinity: NONE

timeoutSec: 30

[robertwolf12]

To resolve the AccessDenied Error (403) without making the bucket public, you should grant access to the load balancer's service account. Here are the steps to find and set up the service account:

1. Find the Load Balancer Service Account:

   • Go to the Google Cloud Console.
   • Navigate to "IAM & Admin" -> "IAM".
   • Look for a service account named something like service-<project-number>@compute-system.iam.gserviceaccount.com.

2. Grant Access to the Service Account:

   • Go to "Cloud Storage" -> "Buckets".
   • Select your bucket.
   • Click on the "Permissions" tab.
   • Add the load balancer's service account and grant it the role "Storage Object Viewer".

This should allow the load balancer to access the bucket without making it public.