Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Composer - Trigger a Dataform Repository with gcp cloud composer

TA3
Bronze 2
Bronze 2


Hi All,

I am performing a Poc to trigger dataform via gcp composer, I am using dataform to perform ETL for a multiple clients depending on the client name i pass as an argument. My requirement is to be able to create a Dag which will trigger the dataform for a client by sending the client_name.

For example 
Below is how my workflow_settings.yaml looks like in a dataform repo, It performs the ETL operations for a client which is sent via code variable, In the below example it would for client A.
================================================

defaultProject: project_id
defaultLocation: europe-west1
defaultDataset: dataform_tm_poc
defaultAssertionDataset: dataform_assertions
dataformCoreVersion: 3.0.0-beta.4
vars:
  code: client A
===========================================

Can anybody please help me in setting up the part of the composer where i can trigger the dataform for a particular client by sending the client code as an argument.
6 1 942
1 REPLY 1