Hello,
We use Airflow on a daily basis but everything runs in a few hours during the morning. After everything ran, we use a pipeline in Gitlab to first take a snapshot of the Composer environment and second, shut it down. The next morning another pipeline starts up the environment and loads the latest snapshot so everything is the same.
The only problem we have is that DAGs run once even with catchup on False and even if they are disabled. For instance, I was testing a DAG every two hours (when Airflow was up of course) and I disabled it a couple days ago. It's still running once a day as soon as Airflow starts.
Same with some DAGs I was testing that should every day (again, catchup in False) but Airflow was running only on weekdays (firsts tests we were doing). On monday those DAGs ran as soon as Airflow started (showing as if they ran the day before - sunday) and then again their normal run at certain time.
Timezone settings are fine and I'm only at UTC-3 but running things mid morning so there's no chance of having trouble there.
Any clues?
Thanks in advance!
LinkedIn
Twitter
