This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
i have some doubts when it comes to choosing between cloud workflows and cloud composers.
In your opinion what kind of situation would cloud workflow not be a viable option? - given the abilities of cloud workflow i feel like it can be used for most of the data pipeline use cases, and I am struggling to find a situation where cloud composer would be the only option.
GCP recommends that we use cloud composer for ETL jobs. Still, at the same time, their documentation on cloud workflows mentions that it can be used for data-driven jobs like batch and real-time data pipelines using workflows that sequence exports, transformations, queries, and machine learning jobs.
Here I am not taking constraints such as legacy airflow code, and familiarity with python into consideration when deciding between these two options with Cloud Scheduler we can schedule workflows to run on specific intervals so not having inbuilt scheduling capabilities would also not be an issue for cloud workflows. So why should I use cloud composer then ?? .
Any real-world examples/use cases/suggestions of why you would choose cloud composer over cloud workflows that would help me clear up the above dilemma would be highly appreciated.
LinkedIn
Twitter
