BigQuery Cross-Regional Replication

[alexdrc00]

Hello,

I am trying to move a dataset from EU to me-central1, I created the replica and tried promoting the secondary to primary as per the docs, I also stopped ingesting data into the dataset in case it causes issues, however the command seems to take forever, it is stuck in 'promoting...' (see example below). I have done all through the console, is it normal for it to be taking so long or any idea what could be causing this?

[caryna]

Hi @alexdrc00,

Welcome to the Google Cloud Community!

According to the documentation, once the secondary replica has fully synchronized with the primary replica, the promotion process should be completed. If the secondary replica does not catch up with the primary, the promotion cannot proceed successfully. 

In your case, me-central1 might still be synchronizing changes from the EU that occurred before you initiated the promotion. 

You should also consider a few key factors that could be contributing to this issue: an unstable connection, resource constraints, and a large dataset. These can all delay replication to the secondary location, as synchronizing changes takes more time under these conditions.

To obtain detailed information about your database replicas, you can check INFORMATION_SCHEMA.SCHEMATA_REPLICAS. Ensure you have the appropriate permissions to access this schema.

For a comprehensive assessment of resource availability and usage, leverage the monitoring tools provided by Google Cloud. These tools can help you track critical metrics such as CPU utilization, memory consumption, disk I/O, and network traffic throughout the promotion process. This will enable you to monitor performance, identify potential bottlenecks, and ensure a smooth transition.

If the issue continues, please contact Google Cloud Support. When reaching out, provide as much detail as possible and include any relevant screenshots of the errors you've encountered. This information will help them diagnose and resolve your problem more effectively.

[jcottrill]

I'm having the same issue, for empty datasets or datasets with a few MB, not a problem, for datasets with GB or TB of data, it freezes. I've followed the documentation exactly as written, and it sticks in pending. A few seconds of downtime is fine, but we have streaming pipelines into our datasets, so we can't deal with a long downtime.  The datasets are synced, but they still fail to promote.

For a comprehensive assessment of resource availability and usage, leverage the monitoring tools provided by Google Cloud. These tools can help you track critical metrics such as CPU utilization, memory consumption, disk I/O, and network traffic throughout the promotion process. This will enable you to monitor performance, identify potential bottlenecks, and ensure a smooth transition.

---

 - This is a useless suggestion, BigQuery is Serverless, we can't do any of that…

There is an email address on the documentation bq-dataset-replication@google.com - But public posting isn't enabled, so emails to it are rejected. 

I'm not paying to contact Google Cloud Support when this is an obvious issue with the product, especially as I work at a Google Cloud Partner agency. Can someone from this team please reach out and I can screen share to show the issue.