Schedule dataform with tags

[alfies02]

I have a set of actions within dataform that need to be run at varying frequencies throughout the day, some hourly and some daily, and there are various dependencies between these. I've trying to set up the schedule at present using dataform and workflows as per the guide here but I'm getting errors (other options to achieve the same result would also be welcome).

The code works fine when running all actions within the main branch but when customizing to execute only actions with a specific tag it throws up a 400 error with the following description:

"description": "Invalid JSON payload received. Unknown name \"invocationConfig\" at 'compilation_result': Cannot find field.",
              "field": "compilation_result"

Just as a side note but I removed the compilation result as it appears this shouldn’t be here and doesn’t allow me to deploy.

Full code attached below:

main:
  steps:
    - init:
        assign:
          - repository: projects/PROJECT_ID/locations/REPOSITORY_LOCATION/repositories/REPOSITORY_ID
    - createCompilationResult:
        call: http.post
        args:
          url: ${"https://dataform.googleapis.com/v1beta1/" + repository +
            "/compilationResults"}
          auth:
            type: OAuth2
          body:
            invocationConfig:
              includedTags:
                - hourly
              transitiveDependenciesIncluded: true
        result: compilationResult
    - createWorkflowInvocation:
        call: http.post
        args:
          url: ${"https://dataform.googleapis.com/v1beta1/" + repository +
            "/workflowInvocations"}
          auth:
            type: OAuth2
          body:
            compilationResult: ${compilationResult.body.name}
        result: workflowInvocation
    - complete:
        return: ${workflowInvocation.body.name}

 We've tried the following:

• capitalising invocationConfig
• Passing included tags directly in the body (without invocationConfig)
• adding IncludedTags as a filter to the URLs. This works for http.get but not for http.post (

  "...dataform.googleapis.com/v1beta1/" + repository + "/workflowInvocations?filter=invocationConfig.includedTags:daily"

  This gives another 400 error, description below:

"message": "Invalid JSON payload received. Unknown name \"filter\": Cannot bind query parameter. Field 'filter' could not be found in request message.", "status": "INVALID_ARGUMENT"​

[aaronscottb]

Just to clarify,  are these three

• capitalising invocationConfig
• Passing included tags directly in the body (without invocationConfig)
• adding IncludedTags as a filter to the URLs

all resulted to the same error "

"message": "Invalid JSON payload received. Unknown name \"filter\": Cannot bind query parameter. Field 'filter' could not be found in request message.", "status": "INVALID_ARGUMENT"

[alfies02 · Accepted solution]

No just the third point resulted in the error.

I actually think I've solved this. It appears that the docs I referred to are incorrect. I think that to run specific tags we should be updating the createWorkflowInvocation block rather than the createCompilationResult. Doing this allows the workflow to be deployed and executed. I've attached a screenshot of the text which I believe needs updating.