Hi there,
We are currently evaluating how to migrate our analytics setup from Scheduled Queries to a Dataform repository. At the moment, we have an SQL workflow that runs as a scheduled query to produce daily snapshots, which are then stored in a historical snapshots table containing data for a specified date range. Additionally, we have a separate query to handle backfilling.
Is it possible to implement a similar design in Dataform? Specifically, how would we handle backfilling the historical snapshot table? I came across the example in the documentation for backfilling an incremental table with JavaScript, but the example provided only covers a simple SQL statement rather than executing an entire SQL workflow over a given time frame.
Would you be able to provide guidance on the best approach for this in Dataform?
Thank you for your help!
LinkedIn
Twitter
