Solved: How to run sequential jobs in Google cloud ( gclou...

gcloudLearning · 05-19-2023 07:26 AM

How to run sequential jobs in Google cloud ( gcloud commands).

I need to run few functions (python based )one by one. please suggest me what is the suggested ways.

KirTitievsky

This very much depends on how long these commands run, how prone they are to failure, how often you plan to run them, and whether they are resource intensive. But here are a couple ways to get us going:

A. There is no shame in spinning up a Compute VM and running the commands in your favorite shell sequentially. You can save the command and code as a startup script so you can reproduce this.

B. If you want full automation, put your python code in Cloud Functions (https://cloud.google.com/functions) and represent your sequence of execution in Cloud Workflows (https://cloud.google.com/workflows)

View solution in original post

KirTitievsky

This very much depends on how long these commands run, how prone they are to failure, how often you plan to run them, and whether they are resource intensive. But here are a couple ways to get us going:

A. There is no shame in spinning up a Compute VM and running the commands in your favorite shell sequentially. You can save the command and code as a startup script so you can reproduce this.

B. If you want full automation, put your python code in Cloud Functions (https://cloud.google.com/functions) and represent your sequence of execution in Cloud Workflows (https://cloud.google.com/workflows)

gcloudLearning

Thank you for the detail. Can we add gcloud commands in cloud function instead of python.

if i need to run , gcloud commands what is the right platform to do so?

KirTitievsky

I haven't been able to find a simple way of doing that, but you can do almost everything gcloud CLI does in python client libraries for individual services you interact with.