Solved: Dataform Upstream table dependencies - Page 2

bihagkashikar · 09-22-2024 05:48 AM

Hello

I am working on a project where a tables `table1`, `table2` is created in BigQuery via terraform.

In dataform, I am building a view based on the table and I want to use this build dependency check in .sqlx file to ensure view is not run before the table is deployed in BigQuery via Terraform. How do I achieve this?

See below query and error.

below sample code of my myview.sqlx file

config { 
type: "view",
schema: constants.dataset_name,
description: "Description of what this model represents"
}

select count(distinct name)
from ${constants.dataset_name}.${ref("table1")} ha
inner join ${constants.dataset_name}.${ref("table2")} sa on ha.account_hkey = sa.account_hkey

Below is the error I am getting

Error: Missing dependency detected: Action my_project_name.my_dataset.myview" depends on "{"name":"table1","includeDependentAssertions":false}" which does not exist

caryna

Hi @bihagkashikar,

You are correct that multiple data sources can be declared within a single JavaScript file. However, querying information_schema to declare sources is not currently supported. Here are some potential approaches you might consider, including the one you've mentioned:

For more detailed guidance, you can refer to best practices for managing data sources.

Alternatively, if needed, you might consider submitting a feature request for enhanced functionality. I can't give a timeline for when this feature will be available, but I suggest checking the issue tracker and release notes for the latest updates.

I hope the above information is helpful.

View solution in original post