Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Dataform: schedule all actions in a schema

Is there a way to schedule all actions in a schema? Note that I specifically don't want to use tags.

Scenario:

  • I have configured Dataform such that all my assertions go to the `assertions` schema (using the `assertionSchema` in `dataform.json`).
  • I want to schedule a workflow that runs the assertions each day: it would be good if I could run everything in the `assertions` schema, without having to explicitly list them out
    • I can't use tags because I have defined a lot of my assertions as part of a table/view config, and so these assertions can't be tagged. It would be a pain to re-write them as individual assertions
  • I use Terraform's  `google_dataform_repository_workflow_config` to define the workflow (link), and I was hoping I could do something like the following (which doesn't work):

 

....
  invocation_config {
    included_targets {
      database = "my-db"
      schema   = "assertions"
      name     = "*"
    }
....

 

  •  But the REST API makes me think that the name has to be an action, not a wildcard or empty)

Any ideas? I'd really like to run my assertions each day in isolation.

0 1 665
1 REPLY 1

The scheduling of all actions in a schema in Dataform without using tags isn’t readily available in standard documentation. While Dataform allows for configuration through the environments.json file, scheduling is typically managed through the Dataform web interface or other external tools. Please refer to the Dataform official documentation for more details