Storage transfer using a destination bucket in another project/account

[juancarlos_la]

I'm looking to create a Storage transfer Job, which uses as a source an S3 bucket, and as a destination, a GCP bucket in a different account/project than the Job is created. Is this something possible to do? If so, what is the process to follow?

[christianpaula]

Hi @juancarlos_la,

Welcome to Google Cloud Community!

Unfortunately, transferring data directly from S3 to a GCP bucket in another project isn't possible with Storage Transfer Service. This might be helpful:

1. IAM permissions:

• Grant read access to the S3 bucket for your GCP project.
• Download data from S3 to a temporary location in your GCP project.
• Transfer data from the temporary location to the target GCP bucket in another project.

2. Third-party tool:

• Use tools like Fivetran, Stitch, or Airbyte to manage data transfer between S3 and GCP.

3. Intermediate bucket:

• Transfer data from S3 to a bucket in your GCP project.
• Then, transfer data from that bucket to the target GCP bucket in another project.

Considerations:

• Permissions, security, and cost implications vary depending on the approach.
• Choose the method that best suits your specific needs and setup.

[juancarlos_la · Accepted solution]

Hi @christianpaula . Thanks for your comments. I solve the issue using this approach:

* In the account where the jobs will be created, I've created a service account.

* In the account with the buckets, I granted these permissions to the service account: storage.buckets.get, storage.objects.create, storage.objects.list, storage.objects.update