FIRESTORE - Import failed

[davidregalado25]

Hello!

I was trying to import some data from a firestore collection to another. In order to achieve this, I used the export feature in project1. Now, I'm inside my project2 trying to do import but with no luck. Here are some screenshots. Am I doing something wrong? 

 

Import screenshot 1

Import screenshot 2

Error screenshot

--
Best regards
David Regalado
Web | Linkedin | Cloudskillsboost

[ms4446]

The error message specifies the path to the file as /boozeco-evidente-test-david/boozeco-evidente-test-david.overall_export_metadata, which seems to be missing or incorrectly specified. Here's a step-by-step approach to troubleshoot this issue:

1. Verify File Existence: Ensure that the file actually exists in the specified location in your Google Cloud Storage bucket. You can check this using the Google Cloud Console or via the Cloud Shell with the command:

   gsutil ls gs://boozeco-evidente-test-david/

   This command lists all files in the bucket. Look specifically for the file named boozeco-evidente-test-david.overall_export_metadata.

2. Check Path in Import Command: Ensure that the path specified in your import command is correct. Based on your message, the path seems to be duplicated (/boozeco-evidente-test-david/boozeco-evidente-test-david.overall_export_metadata). Make sure that the path you use matches exactly with where the file is located in the storage bucket.

3. Use Full Path in Commands: When referencing your file in commands, use the full path from the bucket root, and ensure there are no typographical errors:

   gsutil ls gs://boozeco-evidente-test-david/firestore-exports/

   This can help you verify the file's presence and exact location within subdirectories.

4. Correct Command Format: When importing to Firestore, ensure your command is correctly formatted. Generally, the command to import data from a Google Cloud Storage bucket to Firestore looks like this:

   gcloud firestore import gs://[BUCKET_NAME]/[PATH_TO_EXPORTED_DATA]

   Replace [BUCKET_NAME] and [PATH_TO_EXPORTED_DATA] with your bucket name and the path to the folder containing your Firestore export.

5. Permissions and Access: Verify that your Google Cloud account has the necessary permissions to access the Google Cloud Storage bucket and perform import operations in Firestore.

[davidregalado25]

2. Check Path in Import Command: Ensure that the path specified in your import command is correct. Based on your message, the path seems to be duplicated (/boozeco-evidente-test-david/boozeco-evidente-test-david.overall_export_metadata). Make sure that the path you use matches exactly with where the file is located in the storage bucket.

But that part was generated internally by the console. This is an UI issue.

--
Best regards
David Regalado
Web | Linkedin | Cloudskillsboost

[ms4446]

Thanks clarification. Here are a few steps you can try to work around this problem:

1. Manually Specify the Correct Path: Instead of relying on automatically generated paths or commands, you can manually construct the import command with the correct path to the Firestore export metadata file. Ensure that the path is typed correctly and corresponds exactly to the location and name of the file in your Google Cloud Storage bucket.

2. Use Cloud Shell or gsutil Directly: If the Console UI is causing issues, you might have better success using the Cloud Shell or gsutil directly to perform your operations. You can execute the Firestore import command like this:

gcloud firestore import gs://[BUCKET_NAME]/[PATH_TO_EXPORTED_DATA]

Make sure to replace [BUCKET_NAME] and [PATH_TO_EXPORTED_DATA] with the accurate bucket name and file path.

[davidregalado25]

Hello

It' been a while since a encounter this issue. Today I tried to export and import a database but once again the process failed with the same 404 error.

--
Best regards
David Regalado
Web | Linkedin | Cloudskillsboost

[ms4446]

I'm sorry to hear that you're still experiencing this issue with importing your Firestore database. Try verifying the exact location of the export files in your Cloud Storage bucket using the gsutil command. Run gsutil ls -r gs://boozeco-evidente-test-david/** to identify the presence and path of the overall_export_metadata file. 

Then manually construct the import command to avoid relying on the Console's UI. The import path should point to the directory containing the metadata file, not the file itself.

For example, use gcloud firestore import gs://boozeco-evidente-test-david/firestore-exports/[TIMESTAMPED_FOLDER], ensuring no duplication of the bucket name. If you face issues using the Console UI, executing the command in the Cloud Shell can help bypass UI-related glitches.

Ensure that your account has the necessary permissions, including Storage Object Viewer for Cloud Storage and Cloud Datastore Import Export Admin for Firestore operations. Check the project's IAM policy to confirm these roles.

If the issue persists despite verifying the file path and permissions, try exporting the data again to a new location and reattempt the import. This helps rule out potential corruption or configuration issues with the original export.

If none of these steps resolves the problem, consider updating your gcloud SDK to ensure compatibility and report the issue to Google Cloud Support, providing logs and operation details for further investigation.

[stuart-mesh]

This is no help and I am having the exact same issue, maybe its best to describe how to troubleshoot your permissions issues better. Also realistically all we want as a user is to be able to duplicate our prod database and make a dev database, so 1 click button for this into 2 different projects would be great.

[sumit1826]

One issue I found is if your bucket name will be same as project name it gives error no matter what permission you give, I spent entire day trying to debug and fix the issue. At the end I created a new bucket and it worked without issue. One pattern I am seeing from this post is the name of bucket was same as project. I am thinking some kind of bug in import job with this same name. Try again and let me know if this solved your issue. 🙂