Detect Workflow Configuration on Dataform Run - OR - Run Time

[jaugust-1674854]

I already have a method to ID a Release Configuration (setting a variable in the Release Configuration overrides).

I am looking to find a way to ID the Workflow Configuration (a Schedule) on run. This way I can use different code patterns in files based on which Workflow Configuration is running. Does anyone have a solution?

Alternatively is there a way to get the run-start time when using the built in Workflow configurations instead of Airflow?

[nikacalupas]

Hi jaugust-1674854,

Welcome to the Google Cloud Community!

To identify the Workflow Configuration (Schedule) on run, you can use the environment variables approach. Here’s how you can do it:

• In the Google Cloud Console, go to Dataform  workflow configurations and set a unique environment variable for each workflow configuration. 
• Use dataform.projectConfig.vars.WORKFLOW_CONFIG_ID to access the variable within your SQLX files. This allows you to conditionally execute different code blocks based on the schedule.
• To approximate the run-start time, execute CURRENT_TIMESTAMP() in a very early-running query. 
• Create a simple SQLX file (e.g., start_time.sqlx) that only contains the timestamp query and has no dependencies. This will ensure it runs first.
• Store the result in a temporary view or table and then include it in downstream queries to ensure consistency.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.