Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Error starting Datastream from Postgres to Big query - PostgreSQL backfill permissions - Failed

Hello Datastream Team,

We are trying to use DataStream to replicate data from GCP Cloud SQL PostgreSQL to GCP BigQuery. I've set up the Datastream with the PostgreSQL database as the source profile. All the IPs from the DataStream were added to the Authorized Networks in the GCP Cloud SQL PostgreSQL Instance. We were able to successfully connect. The  Stream, source, and destination connections were created in the same region - us-east4. I validated that:

1) Connectivity tests pass.

2) All except one on the Validation test at the end of the Stream creation wizard were passed.

suryakolachana_0-1694805172596.png

3) I have enabled Enable logical replication on the database.
4) I have c
reated a publication and a replication slot created a datastream user and granted all the sufficient privileges as per the documentation to the schema and tables.

 

Here is the error message I get: 

Failed to start dev

PostgreSQL backfill permissions: Failed

One or more tables in the stream's include list weren't found or Datastream couldn't read from them. Make sure that the tables exist and that Datastream has the necessary permissions.

public. test

Can someone help me troubleshoot the issue?

Thank you.

0 2 1,695
2 REPLIES 2

Did you followed this article for preparing data for postgresql: 

 
It looks like it is missing a permission on the table that is included in the stream. I would suggest filing a support case for this concern for better investigation of your project, data and resources: https://cloud.google.com/contact

Did you ever find a solution for this?  I am experiencing the same.

Top Solution Authors