Hi,
I am using Batch from a little while now. I came across a use case that I would have expect to be covered looking at the API specs but it seems not supported now.
I want to run in the same batch job 2 different tasks with different configuration. I created a Batch job where I have a container (a single runnable in a "taskSpec" definition) running in parallel multiple times and I have another container that needs to be defined with different specs see example below (note that I use Workflow to create a batch job so the example config is in yaml).
taskGroups:
# Call script 1
- taskSpec:
runnables:
- container:
imageUri: ${"europe-west1-docker.pkg.dev/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/my_repo/my_image:latest"}
entrypoint: "/entrypoint_1.sh"
computeResource:
cpuMilli: 2000
memoryMib: 1024
maxRetryCount: 1
maxRunDuration: 1800s
taskCount: 31
parallelism: 31
# Call script 2
- taskSpec:
runnables:
- container:
imageUri: ${"europe-west1-docker.pkg.dev/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/my_repo/my_image:latest"}
entrypoint: "/entrypoint_2.sh"
computeResource:
cpuMilli: 2000
memoryMib: 1024
maxRetryCount: 1
maxRunDuration: 300s
taskCount: 1
parallelism: 1
allocationPolicy:
instances:
- policy:
machineType: e2-standard-2
network:
network_interfaces:
network: ${"projects/" + sys.get_env("GOOGLE_CLOUD_PROJECT_ID") + "/global/networks/my_network"}
no_external_ip_address: false
labels:
env: prod
logsPolicy:
destination: CLOUD_LOGGING
When I try to create this batch job I get this error: "task_groups field is invalid. Only one task group per job is supported error : <nil>".
If I run only one or the other the Batch job complete successfully with no error.
Any plan to support this?
Thank you very much
Best Regards