Best way to add query labels to a model in dataform

[benjikj-cera]

Hi there,

As analytics engineers, we want to be able to tag our dataform queries by the layer and domain so that we can see how much each layer and domain costs us (using the info schema).

We initially thought we could do this via the BigQuery labels config, however, these only populate at the table level, and are very unfriendly to be able to access from the Info Schema and then tag back up to individual queries.

Therefore, we are wanting to apply the query_label reference on each query, which quite simply comes through the labels field in the same info schema table that the query level bytes come through.

The only way we've found that we can do this is by applying a pre_operation, like so:

pre_operations {
    SET @@query_label = "label:mart_core";
}

However, when we do this, the query compilation is affected and dataform is not able to tell us if the query will fail, and also cannot estimate the amount of bytes for the query.

So is there a better way of implementing query_labels to our models that will allow us to still see the query compilation errors and estimated bytes?

Thanks,

Benji