Hello,
Every night, around always the same time (2AM/3AM UTC), our K8s cluster triggers a DataDog alert of overcommitted CPU resources. Funny thing, during that time the data load processing is as low as possible and there is no evidence of high peak load.
The DD alert is based on:
avg(last_10m):sum:kubernetes_state.container.cpu_requested{project:infinity} by {cluster-name} / sum:kubernetes_state.node.cpu_capacity{project:infinity} by {cluster-name} > 1
Any idea about this behaviour?
Thanks