How to add Cloud SQL proxies as resources to a Batch Job

[bsdz_ci]

Not sure which board to post this on.

I've a regular batch job that runs on my local workstation. It requires GPU resources as well as access to several SQL databases. One of those databases is already hosted on Cloud SQL. I plan to move the other to Cloud SQL too.

Currently, I use Cloud SQL Proxy in a container (within same docker-compose network) to connect to my Cloud SQL instance.

I'd like to migrate the workstation job to Google Batch. I can't find any examples of how to add Cloud SQL Proxy resources to a Batch Job. I'd like to avoid adding Cloud SQL Proxy code inside my actual worker containers; that said, if it's unavoidable then that's the way it is.

Any recommendations?

[Marramirez]

Hi @bsdz_ci,

Welcome to the Google Cloud Community!

You can refer to this documentation.

Here, provides an example on how to create a job that uses a GPU. To add a GPU to a job, you can use different types of runnables you want to use:

• Add a GPU to a container job.
• Add a GPU to a script job.
• Add a GPU to a container and script job

Hope this helps!

[bsdz_ci · Accepted solution]

Hi @Marramirez . Thanks for the suggestions. I think by mentioning GPUs in my question I must have confused you. I was asking how to add a Cloud SQL Proxy to my batch jobs. Eventually I tried adding a runnable container with the google sql proxy docker container (setting background to true and adding a barrier) but I couldn't get it to work. I resorted to moving the proxy connection code into my application (inside the worker containers) and this appears to work. 

[nikodraca]

Hi @bsdz_ci, do you have a code snippet of how you set this up? I can't seem to figure this out

[mauronr]

@bsdz_ci I'd appreciate if you could share any pointers of how you managed to get it to work too.