Solved: Re: sync local folder and vm disk

JakubMituraa · 07-03-2022 10:06 AM

My goal is to synchronise data in vm with my local folder

First I list my vm disks

gcloud compute disks list

I get between others

nvidia-ngc-image-vm-kuba

I create local directory

 /media/jakub/NewVolume/dataForVm

I can successfully ssh from gcloud on my local computer to the instance associated with disk

Now I try to synchronize folders

gsutil -m rsync -r /media/jakub/NewVolume/dataForVm nvidia-ngc-image-vm-kuba

I get

CommandException: arg (nvidia-ngc-image-vm-kuba) does not name a directory, bucket, or bucket subdir.

adding specific directory on the disk do not make a difference

gsutil -m rsync -r /media/jakub/NewVolume/dataForVm nvidia-ngc-image-vm-kuba/data/

Thanks for any help!

alexmoore

Hi there,

One thing to keep in mind is that there are different types of storage in Google Cloud. A local disk for a Virtual Machine is known as a Persistent Disk or PD (https://cloud.google.com/persistent-disk) - this is a block storage technology and in general only available directly from the Virtual Machine to which it is connected. There is also Google Cloud Storage or GCS (https://cloud.google.com/storage), this is an object storage technology available via API that, depending on permissions, can be accessed remotely in many scenarios.

The "gsutil" command you are using would indeed be able to synchronise data from a local directory, but it is specifically for interacting with Google Cloud Storage and would not be able to send data directly to the Persistent Disk of your VM.

Given you have SSH access to the VM, my suggestion would be to explore tools like "rsync" which can be used to synchronise files and folders over an SSH connection - thereby allowing you to write via SSH directly to the attached Persistent Disk. This guide seems like a reasonable introduction to using the tool, though there are many others: https://linuxize.com/post/how-to-use-rsync-for-local-and-remote-data-transfer-and-synchronization/

Hope that helps,

Alex

View solution in original post

alexmoore

Hi there,

One thing to keep in mind is that there are different types of storage in Google Cloud. A local disk for a Virtual Machine is known as a Persistent Disk or PD (https://cloud.google.com/persistent-disk) - this is a block storage technology and in general only available directly from the Virtual Machine to which it is connected. There is also Google Cloud Storage or GCS (https://cloud.google.com/storage), this is an object storage technology available via API that, depending on permissions, can be accessed remotely in many scenarios.

The "gsutil" command you are using would indeed be able to synchronise data from a local directory, but it is specifically for interacting with Google Cloud Storage and would not be able to send data directly to the Persistent Disk of your VM.

Given you have SSH access to the VM, my suggestion would be to explore tools like "rsync" which can be used to synchronise files and folders over an SSH connection - thereby allowing you to write via SSH directly to the attached Persistent Disk. This guide seems like a reasonable introduction to using the tool, though there are many others: https://linuxize.com/post/how-to-use-rsync-for-local-and-remote-data-transfer-and-synchronization/

Hope that helps,

Alex