Data Transfer to Cloud bucket creates unwanted folders

[Rob]

I'm using data transfer to load files referenced in a tsv file on my own server.  The destination is a folder within a bucket.  I would expect it to put the retrieved file directly in that folder, but instead it creates a new folders underneath my bucket destination folder.

Example:

TSV file contains:

https://www.mydomain.com/dir1/dir2/filename.doc

Destination:

MyBucket

Folder:

MyFolder

After running the data transfer, it creates under "My Folder" a sub folder called "www.mydomain.com" and under that "dir1" and under that "dir2".  

So instead of accessing my file at:

bucket.storage.googleapis.com/dir1/dir2/filename.doc

The URL ends up being:

bucket.storage.googleapis.com/dir1/dir2/www.rustyzipper.com/dir1/dir2/filename.doc

I've looked through the docs & I can't find anything that explains this behavior or how to fix it.

Help!?

Thanks!

Rob

[Darrell_M]

Hi,

This is the intended behavior as the object name is actually "www.rustyzipper.com/dir1/dir2/filename.doc". Do note that Cloud Storage operates with a flat namespace, which means that folders do not actually exist within Cloud Storage[1]. Futhermore, This section[2] explains the behavior you are experiencing: 

"When an object located at http(s)://[HOSTNAME]:[PORT]/[URL_PATH] is transferred to Cloud Storage, the name of the object in Cloud Storage is [HOSTNAME]/[URL_PATH]."

I hope this clears up the confusion. 

[1] https://cloud.google.com/storage/docs/folders

[2] https://cloud.google.com/storage-transfer/docs/create-url-list#url-list-format