Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Support for multiple "tasksSpec" within "taskGroups"

Hi,

I am using Batch from a little while now. I came across a use case that I would have expect to be covered looking at the API specs but it seems not supported now.

I want to run in the same batch job 2 different tasks with different configuration. I created a Batch job where I have a container (a single runnable in a "taskSpec" definition) running in parallel multiple times and I have another container that needs to be defined with different specs see example below (note that I use Workflow to create a batch job so the example config is in yaml). 

 

 

taskGroups:
# Call script 1
- taskSpec:
    runnables:
    - container:
        imageUri: ${"europe-west1-docker.pkg.dev/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/my_repo/my_image:latest"}
        entrypoint: "/entrypoint_1.sh"
    computeResource:
      cpuMilli: 2000
      memoryMib: 1024
    maxRetryCount: 1
    maxRunDuration: 1800s
  taskCount: 31
  parallelism: 31
# Call script 2
- taskSpec:
    runnables:
    - container:
        imageUri: ${"europe-west1-docker.pkg.dev/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/my_repo/my_image:latest"}
        entrypoint: "/entrypoint_2.sh"
    computeResource:
      cpuMilli: 2000
      memoryMib: 1024
    maxRetryCount: 1
    maxRunDuration: 300s
  taskCount: 1
  parallelism: 1
allocationPolicy:
  instances:
  - policy:
      machineType: e2-standard-2
  network:
    network_interfaces:
      network: ${"projects/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/global/networks/my_network"}
      no_external_ip_address: false
labels:
  env: prod
logsPolicy:
  destination: CLOUD_LOGGING

 

When I try to create this batch job I get this error:  "task_groups field is invalid. Only one task group per job is supported error : <nil>".

If I run only one or the other the Batch job complete successfully with no error.

Any plan to support this?

Thank you very much

 

Best Regards

0 6 931