Batch - using instance templates
Hello,We're trying to set up a GCP batch processing with a custom instance template but we hit an issue.The pr...
Forum Posts
Setting permissions for mount point
I'm using a persistent disk mount with:"volumes": [ { "deviceName": "scratchSSD", "mountPath": "${workDir}", "...
gcloud cli sometimes unavailable on Compute Engine VM during batch job
I've created an image of a disk that I manually setup, it's an Ubuntu 20.04 install, I have gcloud working, an...
Batch - Newly created job isn't starting? Stuck in Queued status?
For some reason this morning several Batch jobs which I've attempted to create are getting stuck and not getti...
Which tool or GCP service helps to ssh in each VMs of a MiG and execute any shell script?
Hi,I am looking for an advice to use which service or tool (like cloud scheduler or cloud composer), which wou...
Batch: long wait until job starts
When creating a cloud batch job, it appears SCHEDULED in the list for a few minutes. Even when resources are a...
Batch - Creating new disk fails when using snapshot
I made a job that mounts a new disk and it worked well. But when I specify a snapshot for that disk, the job f...
Batch - Specify image-family
Is there any way to specify the image family for a job in the `InstancePolicy` alongside `machineType`?Am I co...
BATCH - How to manage number of instances
Hi all,I have enough CPU quota.When running a job of 22 tasks (~10 min to complete each):- Parallelism = 22- I...
Batch - Issues mounting new disk
I am trying to create a job through python, this is my creation script: def submit_download_sra_fastq(sra_batc...
Batch - Increasing size of boot disk
I am creating jobs to submit to Batch through the python sdk. I am running a docker container on many instance...
Batch - User Segmentation
Hi. Is there any sense of segmentation by user? For example, have multiple users in a project. Each user is do...
No logs or error messages available to troubleshoot 'Failed' job
I am trying to create a Batch job that mounts a GCS bucket.I have configured logs to go to cloud logging.For t...
How to extract data from Cloud SQL (sql server) with Dataflow to BigQuery?
Hi team, I am trying to send the CDC (Change Data Capture) from a cloud sql server instance to bigquery. I man...
Batch - Mounting Existing Persistent Disk
Hi. I'm having trouble mounting an existing persistent disk using batch. I get the following errormount: /mnt/...
Is there a way to get the general availability of a certain predefined machine type for spot vms?
Let's say my task is embarrassingly parallel and I have set parallelism to 1,000. I can run my program twice a...
Support for multiple "tasksSpec" within "taskGroups"
Hi,I am using Batch from a little while now. I came across a use case that I would have expect to be covered l...
Scheduling a Google Batch job to trigger on GCP (long running Python script)
By "Google Batch" I'm referring to the new service Google launched about a month or so ago.https://cloud.googl...
Batch - TPU Support
Hello everyone,Is there already a way for running Jobs on TPUs using Batch? If not, do you know if TPU support...
Batch Job + VPC Connector
Hey, is there a way to indicate which VPC connector should be used by the Batch job?I am using the new Batch A...
Batch in GCP New Feature
https://cloud.google.com/blog/products/compute/new-batch-service-processes-batch-jobs-on-google-cloud?utm_sour...
Infrastructure: Compute, Storage, Networking
This is the discussion space to talk about all things Infrastructure: Compute, Storage, Networking related.