Solved: Scheduling runs in Dataform / BigQuery - Page 2

aboudella · 10-29-2022 04:06 AM

Hello,

Is there a way to schedule runs directly from Dataform with an environments.json file as described in the doc here ? Or do I need to use other Google Cloud services like Scheduler/Workflows or Composer ?

Thanks in advance for your reply.

lewish

schedules.json and environments.json are only read by "legacy" Dataform - Dataform on GCP does not understand them.

You are correct in thinking that currently the only way to schedule Dataform workflows on GCP is with Workflows/Composer or similar - https://cloud.google.com/dataform/docs/schedule-executions-workflows

Native scheduling as part of Dataform on GCP is in the pipeline and will be available before GA.

View solution in original post