Alternative to GCP Workflow

[KeiwoKeiwo]

Greetings,

Scenario: file land in storage bucket, if file name has pattern *fileName1 then move file to storage_bucket1 else move to storage_bucket2.

What alternative exist for GCP Workflows. 

Note: still new to GCP world and I am experimenting. 

Thank you 🙂

[lawrencenelson · Accepted solution]

Hi @KeiwoKeiwo,

Welcome to the Google Cloud Community!

In your scenario, you may utilize Google Cloud's Cloud Functions [1].

With Cloud Functions, you write simple, single-purpose functions that are attached to events emitted from your cloud infrastructure and services. Your function is triggered when an event being watched is fired. Your code executes in a fully managed environment. There is no need to provision any infrastructure or worry about managing any servers [1].

Cloud Functions provides a connective layer of logic that lets you write code to connect and extend cloud services. Listen and respond to a file upload to Cloud Storage, a log change, or an incoming message on a Pub/Sub topic [2].

For a list of supported programming runtimes. You may access this documentation.

Here is a sample code written in Python for the scenario that you provided:

from google.cloud import storage
import os

def move_file(event, context):
    """Triggered by a change to a Cloud Storage bucket.
    Args:
        event (dict): Event payload.
        context (google.cloud.functions.Context): Metadata for the event.
    """

    file_name = event['name']
    bucket_name = event['bucket']
    storage_client = storage.Client()

    source_bucket = storage_client.bucket(bucket_name)
    source_blob = source_bucket.blob(file_name)

    if "fileName1" in file_name:
        destination_bucket_name = 'storage_bucket1'
    else:
        destination_bucket_name = 'storage_bucket2'

    destination_bucket = storage_client.bucket(destination_bucket_name)
    blob_copy = source_bucket.copy_blob(source_blob, destination_bucket, file_name)
    source_blob.delete()

    print(f'File {file_name} moved to bucket {destination_bucket_name}')

I hope this helps. Thank you. 😃

[1]. https://cloud.google.com/functions/docs/concepts/overview#functions
[2]. https://cloud.google.com/functions/docs/concepts/overview#connect_and_extend_cloud_services