I'm trying to find the best way to communicate between 2 containers (cont1 and cont2) whenever a file is uploaded. I'm trying to achieve and automate this simple architecture:
A file is uploaded in cont1 => cont2 gets the file and does some processing => cont1 gets results back.
The idea was to use PubSub for that:
1) A container stores a file to Google Cloud Storage and publishes a message containing the relevant info to retrieve this file.
2) In another container some subscriber listens, and gets the file from Google Cloud Storage based on the message content.
3) This file is read and its content is processed and the results are published into some subscription.
Reading some docs on the internet, I feel like it is not the best approach to complete this task. My question, is it really the right way to go?
If yes:
1) In order to collect and use the information passed in the message, I had to use sync pull instead of async pull used in this documentation. https://cloud.google.com/pubsub/docs/pull#synchronous_pull It seems that it is not possible to collect the content of a message using a callback function. Is there a workaround?
2) Would the usage of a "push subscription" instead of "pull subscription" be better in order to automate the process? What would be the endpoint in the case of two containers running on the cloud ? doc: https://cloud.google.com/pubsub/docs/samples/pubsub-create-push-subscription.
Thank you very much and I'm looking forward to your feedback.
LinkedIn
Twitter
