GCP Datastream doesn't replicate new tables from RDS Postgres

[fred-egmorte]

Hello,

I have a datastream that replicate RDS postgres to Bigquery dataset.

Everything is set correctly, the datastream works. But we add new tables in RDS postgres and we don't see these new tables neither in the datastream nor in bigquery dataset.

We tried to add the new tables in publication, recreate publication, even recreate publication_slot but the new tables never showed up in bigquery.

Any ideas about how to handle new tables from RDS source to bigquery dataset ?

Thanks a lot.

[nceniza]

I am not sure what steps you have taken for establishing your connection, are you following these instructions perhaps? if not please try these steps included in these.

https://cloud.google.com/datastream/docs/configure-your-source-postgresql-database#rdsforpostgresql

[fred-egmorte]

Hello,

Yes i followed this step. And i even created a medium post with my step by step process:

https://awstip.com/aws-rds-postgres-replication-to-gcp-bigquery-with-gcp-datastream-4ad7653dc246 

I got things a bit further by testing some changes on RDS. And It seems I have to recreate either the publication and the replication slot or just the replication slot to see the new table created on bigquery. Now i am able to see one new table in bigquery.