Here's my scenario: I'm trying to setup replication between a GCS bucket in Montreal and Toronto via Cloud Functions. Actually, I've managed to do that already: I run 2 functions replicate_to_tor_gcs and delete_from_tor_gcs and they work as expected. I'm using the triggers google.storage.object.finalize and google.storage.object.delete respectively.
The problem occurs when I delete the source (Montreal) bucket entirely and I guess because it deletes the individual objects within the bucket first before the bucket itself (or at least it sends the event trigger for each object that was deleted along with the bucket) and so now the mirrored contents in my destination (Toronto) bucket are also deleted.
The reason why this is a problem: there's no dual or multi-region bucket replication available in Canada and because my clients are in regulated industries, their data needs to stay in Canada (i.e. I can't use dual or multi-region buckets in the US) and I need a way to replicate storage. If a region were to go down, would that trigger a delete event since the files don't exist anymore? I'm guessing not, but what if someone did something malicious or accidentally deleted the source bucket?
Has anyone here had to deal with DIY GCS bucket replication via Cloud Functions? If so, how did you deal with bucket deletes?
LinkedIn
Twitter
