Accessing Multi region/location datasets in Dataform

[bihagkashikar]

I have a use case where dataform codebase repository is hosted in EU region. Under this there are several workspaces. Within these workspaces, I need to work with the dataset which is EU region as well as in eu-west1 region. In the dataform.json has defaultLocation = EU. I have separate .sqlx files which work on data in these separate regions. The issue I am getting is I am not able to access the data from EU given the defaultLocation property overrides it. 

Question is there any way in which we could work with multi location datasets by overriding the defaultLocation property in dataform.json

[Aris_O]

Hi @bihagkashikar,

Welcome back to Google Cloud Community,

Transfers from Cloud Storage to BigQuery require that the Cloud Storage bucket be colocated with the BigQuery destination dataset.

This article might help you:
https://cloud.google.com/bigquery/docs/move-datasets
https://cloud.google.com/bigquery/docs/dts-locations

[bihagkashikar]

Hi @Aris_O 

thank you for the links. I agree, moving data is one option however if I am not mistaken, if you consider "other" google services that operate with data like Cloud function, dataproc, dataflow - these allow to work on data multi region so why not dataform. I am curious. hope this help give an perspective.