Storage Transfer Service - copies only one of non-current versions given in manifest

[igor-wo]

Hi,

I am preparing POC for migrating blobs between GCS buckets. We would like to migrate all versions so I:

1. enabled versioning on the target bucket
2. prepared 2 manifests as given in https://cloud.google.com/storage-transfer/docs/cloud-storage-to-cloud-storage#handle_versioned_objec...
3. Copy these manifests to GCS

cloud storage ls --all-versions --recursive --json gs://mig-test-bkt-source > object-listing-gcloud.json
jq -r '.[] | select( .type=="cloud_object" and (.metadata | has("timeDeleted") | not)) | [.metadata.name, .metadata.generation] | ' object-listing-gcloud.json > live-object-manifest-gcloud.csv
jq -r '.[] | select( .type=="cloud_object" and (.metadata | has("timeDeleted"))) | [.metadata.name, .metadata.generation] | ' object-listing-gcloud.json > non-current-object-manifest-gcloud.csv
gsutil cp live-object-manifest-gcloud.csv gs://mig-test-manifest-bkt/live.csv
gsutil cp non-current-object-manifest-gcloud.csv gs://mig-test-manifest-bkt/non-current.csv

 When I try to use the manifest for transfer it fails after first version. When there are 3 non-current versions, only one is copied and then job fails with ALREADY_EXISTS error. However, if I re-run the job, it copies another version and then throws FAILED_PRECONDITION error (but we have progress with copied versions - see screenshot below).

I tried to run transfer with manifest both with Python libraries and in GCP console and the result is the same. My non-current manifest:

Results in console:

Should I add some additional transfer parameters? Or reformat the manifest somehow?