Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Working with multiple repositories in GCP Dataform

Hello there,

We originally had one Dataform repository but due to the large number of Dataform actions and compilation errors, we decided to split it to two repositories (repo_1 and repo_2). The split happened based on the third suggestion on this official document:

-> "1 central repository and 1 repository per domain that uses the contents of the central repository as data sources."

However, the split has increased the development effort significantly, mainly due to the fact that if a change occurs in repo_1, developer has to

  1. Compile repo 1
  2. Trigger execution of the new object so it is created
  3. Declare the new source in repo_2

While I am aware there is an open issue on Google issue tracker for this, I was wondering if anyone else is having this issue, or have tried to address it somehow? For example, is there any way that repo_2 would always be aware about all its sources in repo_1?

Thanks for your help in advance!

Azade

2 1 221
1 REPLY 1

Hello,

Thank you for contacting Google Cloud Community !

As mentioned in the Public issue tracker ,we do not have any ETA for this issue. Please monitor the Issue tracker in order to get updates.

 

Regards,

Jai Ade