Hey, I have several dataflow batch pipelines running once per day to read data from MongoDB and ingest it into BigQuery (I use Dataflow template for that). But for some reason sometimes some pipelines do not start at all. I just don't see any information about running jobs on that day (it's neither Succeeded, not Failed, they just didn't start). Please advise on how to troubleshoot that.
UPD: for scheduling I am using daily dataflow type. Maybe it’s buggy and makes sense to try custom
LinkedIn
Twitter
