One of my TPU-VMs can't Write to Bucket

[themoxon]

I have a number of Google cloud TPU-VMs which need to write to a Bucket.

Most of them work fine, but the one I created this morning is giving me Access Denied when I run gsutil -m cp test_file.txt gs://MY_BUCKET_NAME:

  "error": {
    "code": 403,
    "message": "Access denied.",
    "errors": [
      {
        "message": "Access denied.",
        "domain": "global",
        "reason": "forbidden"
      }
    ]

The VM is identical to all the others as far as I can see (same project, same versions, same region, same service account, gcloud config list and gcloud auth list give identical results) - any ideas why it's behaving differently?

This still happens even if I give allUsers public permissions to write to the bucket.

[Andre_Fiesco]

Checking Google's documentation, I found out that the error means the following: 

cloudstorage.ForbiddenError

This error (403) indicates that the user was not authorized by Google Cloud Storage to make the request.

• The various possible causes for this error are listed in the Google Cloud Storage error documentation for 403-Forbidden.
• A common source of this error is that the bucket permissions (bucket ACL) are not set properly to allow your app access. See Google Cloud Storage Authentication for information on setting up access.

403—Forbidden

Error Name	Description
AccessDenied	Access denied. This applies to any object you don't have access to, regardless of whether or not the object actually exists.

The VM probably is identical but does not have the same scopes:

For example, application A with an access token with read-only scope can only read, while application B with an access token with read-write scope can read and modify data. Neither application can read or modify access control lists on objects and buckets; only an application with full-control scope can do so.

Type	Description	Scope URL
read-only	Only allows access to read data, including listing buckets.	https://www.googleapis.com/auth/devstorage.read_only
read-write	Allows access to read and change data, but not metadata like IAM policies.	https://www.googleapis.com/auth/devstorage.read_write
full-control	Allows full control over data, including the ability to modify IAM policies.	https://www.googleapis.com/auth/devstorage.full_control

Try stopping one of the VMs and providing read-only and read-write scopes.    

Also try the following in case you get the same error (it could be because the correct API is not enabled). In that case, follow these steps and try writing something to the bucket:

Stop VM

1. Stop VM
2. goto --> VM instance details.
3. in "Cloud API access scopes" select "Allow full access to all Cloud APIs" then Click "save".
4. restart VM and Delete ~/.gsutil