Importing and exporting files between different Google Cloud Platform (GCP) environments.

[aka251]

Hi there,

I am trying to move a specific Firestore collection to Firestore in a different GCP environment. I was able to export it without any issues, but I'm not sure how to import it into Firestore in the different GCP environment.

I referred to the following two documents, but they were not helpful as their purpose was different and they were quite confusing.

1. https://cloud.google.com/firestore/docs/manage-data/export-import
2. https://cloud.google.com/firestore/docs/manage-data/move-data

it's possible to request the necessary permissions from the owner in the target GCP environment for the import.

Thanks.

[ms4446]

To import a Firestore collection into a different GCP environment, you can follow these steps:

1. Create a Cloud Storage bucket in the target GCP environment.
2. Give the Cloud Firestore service agent in the source GCP environment permission to read from the Cloud Storage bucket in the target GCP environment.
3. Upload the exported Firestore collection files to the Cloud Storage bucket in the target GCP environment.
4. Start an import operation in the target GCP environment,specifying the path to the exported data in the Cloud Storage bucket.

Here are the specific commands to perform each step:

To create a Cloud Storage bucket:

gcloud storage buckets create <bucket-name>

To give the Cloud Firestore service agent permission to read from the Cloud Storage bucket:

gsutil iam ch serviceAccount:service-PROJECT_NUMBER@gcp-sa-firestore.iam.gserviceaccount.com:roles/storage.legacyBucketReader,roles/storage.objectViewer gs://<bucket-name>

To upload the exported Firestore collection files to the Cloud Storage bucket:

gsutil cp -r <local-directory>/* gs://<bucket-name>/<path-to-exported-data>

To start an import operation:

gcloud firestore import gs://<bucket-name>/<path-to-exported-data>

For example, if you exported a Firestore collection named products to a local directory named /tmp/products, and the exported data has a timestamp in its directory name, you would use the following commands to import it into a Cloud Storage bucket named my-bucket in the target GCP environment:

gcloud storage buckets create my-bucket
gsutil iam ch serviceAccount:service-PROJECT_NUMBER@gcp-sa-firestore.iam.gserviceaccount.com:roles/storage.legacyBucketReader,roles/storage.objectViewer gs://my-bucket
gsutil cp -r /tmp/products/* gs://my-bucket/products/2023-10-24T12:34:56_789Z
gcloud firestore import gs://my-bucket/products/2023-10-24T12:34:56_789Z

Once the import operation is complete, the products collection will be available in Firestore in the target GCP environment.

Clarifications:

• The gcloud firestore importcommand requires the path to the exported data in the Cloud Storage bucket, not just the bucket name. Ensure you specify the full path, especially if the exported data directory includes a timestamp.
• The Firestore service account also needs the roles/storage.objectViewerrole to read the exported files from the Cloud Storage bucket.
• The gsutil cp command requires the gs:// prefix for the Cloud Storage bucket path.

Additional Notes:

• When using the Firestore Admin SDK or any other SDK, ensure you authenticate using a service account with the necessary permissions. This service account should have permissions to read from the source Firestore database and write to the target Firestore database. However, note that the Firestore Admin SDK does not have direct methods like export_documents or import_documents. Ensure you're using the appropriate methods or tools for your operations.