Access Denied Error when Uploading Files from VM to Cloud Storage

[Dhruv05]

I’m trying to upload files from a VM instance to a Google Cloud Storage bucket using the gcloud storage cp command. I’ve attached a service account to the VM with Storage Admin access, but I’m encountering an access denied error.

`gcloud storage cp temp_files/test.txt gs://test-us-east/test.txt`

Error message:

```

Copying file://temp_files/test.txt to gs://test-us-east/test.txt

ERROR: User [vm-instances-sa@gcpplatform.iam.gserviceaccount.com] does not have permission to access b instance [test-us-east] (or it may not exist): Access denied.

Completed files 0/1 | 0B

```

I’ve verified that the service account has the necessary permissions to create objects in the bucket using the Policy Analyzer. Despite temporarily granting Owner access, the issue persists.

I also checked the permissions tab under my bucket, and it shows my service account with storage admin permissions. Additionally, I reviewed the access scope for my VM, and it allows access to all APIs.

[KyleMari · Accepted solution]

Hi @Dhruv05,

Welcome to the Google Cloud community!

I understand that the service account is still unable to gain access to your "test-us-east" bucket after verifying through the Policy Analyzer that it already contained the necessary permissions to be able to upload your file successfully from your virtual machine (VM) to Cloud Storage bucket. 

Even though "Storage Admin" alone should already be good enough to meet the required permissions, it's still worth a try if you can grant the service account additional roles like "Storage Object Creator" and "Storage Object Admin". 

After that, take note of the following methods below which may help resolve the issue:

• Test by restarting your VM: Given that the access scope for your VM has already been set to “Allow full access to all Cloud APIs”, there are instances wherein restarting the VM may refresh the attached service account and the roles they're associated with. Creating another VM may also yield a different behavior but it's best to consider this step as a last resort. All of these tips are related to the similar discussion from Stack Overflow.
• Make sure the bucket name is spelled correctly: This has also been my issue when I was reproducing this, but I later on realized that this can be a huge factor when performing an upload. After seeing to it that the three roles above have been granted to the service account, and the bucket names specified from both sides (VM/CLI and GCP console) are exactly matched, the upload attempt using your same command worked on my end.

If the issue persists, I'd recommend filing a customer issue in the public issue tracker for Cloud Storage. This will allow the Google Cloud team to further investigate any issues behind performing an upload using a service account from the VM. Note that the timeline as to when they’ll be able to resolve any filed issues is something I won’t be able to provide.

Hope this helps!

[Dhruv05]

Hey @KyleMari , removing all permissions from the service account and granting only storage admin access, along with full API access for Cloud Scope, ultimately worked. I stopped the current VM and restarted it, and now the `gcloud cp` command is functioning properly.

[alberto255]

Hello, I encountered the same issue.

I set up a Google Cloud VM instance where I needed to use CLI (shell) commands to transfer a file from the VM to a Cloud Storage bucket. However, I ran into a permission issue with the default service account assigned to the VM during its creation. Despite assigning the service account various roles, including:

- Compute Storage Admin
- Storage Admin
- Storage Object Admin
- Storage Object Creator
- Storage Folder Admin
- Storage Insights Collector Service
- Storage Object User
- Storage Object Viewer

None of these roles worked when using the CLI. The only solution that worked for me was using a 'Service Account Key,' which is a JSON key to authenticate and provide access to the service account.

I also managed to get it working by using the command `gcloud auth login`, which required authentication with my user account. Only then was I able to successfully transfer the file.