How to do i setup cross-project CDC using DataStream - Do i need VPN, Interconnect etc???

[dheerajpanyam]

I have a Cloud SQL Postgres residing in a GCP project , let's call it project-A. The destination is a BQ dataset residing  in a different GCP project, let's call it project-B. I totally understand VPC peering is needed to setup CDC via  the Datastream managed service in GCP with source as Postgres and destination as BQ in this case but the real question is do i need a VPN or Interconnect, documentation is confusing. I am assuming VPN or Interconnect is only needed if the source DB is residing on-premises.

I also see that for VPC peering with private connectivity there is a need for a NAT VM as per the documentation and i don't understand the need for it. By virtue of the VPC peering connectivity within DataStream doesn't cloud sql instance have connectivity by default? Or is it because Cloud SQL instance resides in a Google managed VPC and the VPC peering gives access to resources deployed in the user defined VPC only and hence the need for VM?

[kensan]

Hi @dheerajpanyam,

Welcome to Google Cloud Community!

If you're both Project is within the Google Cloud then you don’t need to use the interconnect or VPN. Since VPN or Interconnect is generally used for connecting on-premises networks to Google Cloud VPC.

Cloud SQL instances are hosted in Google-managed VPCs, which are generally isolated from user-defined VPCs. Although the Cloud SQL instance is within Google Cloud, it resides in a Google-managed network and doesn’t have automatic connectivity to user-defined VPCs, like the one where your Datastream service is likely running.

VPC Peering between project-A where Cloud SQL is located and project-B where Datastream or other resources are hosted allows communication between resources in both VPCs. However, since Cloud SQL resides in a Google-managed VPC, a NAT VM is often needed to route traffic from the user-defined VPC in project-B to Cloud SQL in project.

I hope the above information is helpful.

[dheerajpanyam]

Thanks @kensan for your reply. In my case Cloud SQL, NAT VM and DataStream are in Project A and only BQ is in Project B. Also I had another very important question. Is BQ in destination project supported. Looking at its connection profile I don’t see the connection type option. Had it been there I would have used the IP_PEERING option to indicate BQ is in the remote project