Solved: Is Cloud bucket to Cloud bucket transfer faster/sl... - Page 2

savetsa · 03-18-2024 07:24 AM

Q1.) when using gcloud storage cp/mv/rsync, which is the better option for a initial upload..?
CloudBucket1 -> CloudBucket2 or On-Premise (LocalWorkstation) -> CloudBucket2

Q2) Is mv faster than cp/rsync while uploading?

I am asking this with respect to a folder with 850,000 small files of sizes (10-500kb each). I have to upload this to my bucket and looking for the fastest way to do it...

Please help! @DamianS @cesan3 @iamawaneendra @juancarlos_la @EnterSecurity

DamianS

Ad1). Basically, if you dealing with data around the cloud, afair, bucket is doing:
1. Decryption at rest for your data in bucket1
2. Copy / rsync data to bucket 2 AND
3. Encryption at rest for your data in bucket2
All within storage, so it might be true, that OnPrem to Bucket should be faster, because Encryption at rest for data taken from OnPrem is only when data is being transfered either by gcloud or gsuitl command ( or via UI ). So for me, there is less steps to perform, that in bucket2bucket case.

Ad2. Both are great. gsutil is more advanced as he has parallel synchronization and more granular control over behavior. So if you are using for daily basis gcloud commands, use gcloud storage rsync, but if you prefer dedicated cmd for buckets, use gsuitl 🙂

cheers,
DamianS

View solution in original post

Is Cloud bucket to Cloud bucket transfer faster/slower than On-Prem to Cloud Bucket?