in GKE, K8s components kubedns is not added to the K8s endpoint object

[ramumangalarapu]

Hi Team, 

I am trying to fetch prometheus metrics of the DNS of the managed GKE k8s cluster, and my Prometheus scrape config is based on endpoints; in bare-metal and k3s, I can see the 9153 has been added to the k8s endpoint object but not in GKE, please find below.

            - job_name: "core-dns"
              kubernetes_sd_configs:
                - role: endpoints
              relabel_configs:
                - source_labels:
                    - "__meta_kubernetes_namespace"
                    - "__meta_kubernetes_service_name"
                  action: keep
                  regex: "kube-system;(coredns|kube-dns)"

                - source_labels: [__meta_kubernetes_endpoint_port_name]
                  action: keep
                  regex: "metrics"

                - source_labels:
                    - "__meta_kubernetes_endpoint_port_name"
                  action: replace
                  replacement: "9153"
                  target_label: "__meta_kubernetes_endpoint_port_name"
                  regex: ".*"

apiVersion: v1
kind: Endpoints
metadata:
  creationTimestamp: "2024-07-12T13:58:59Z"
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: KubeDNS
  name: kube-dns
  namespace: kube-system
  resourceVersion: "16842612"
  uid: cf14d8ad-0d00-4704-b33e-2b7ef52aad9a
subsets:
- addresses:
  - ip: 10.44.0.6
    nodeName: my-node-name
    targetRef:
      kind: Pod
      name: kube-dns-66cb54875f-2hgjk
      namespace: kube-system
      uid: 4095797d-3ac9-404f-b066-3b4eef0746a2
  - ip: 10.44.1.5
    nodeName: my-node-name
    targetRef:
      kind: Pod
      name: kube-dns-66cb54875f-knx9c
      namespace: kube-system
      uid: 3ba63667-4f0f-4ed7-8cbb-f75f6454d474
  ports:
  - name: dns-tcp
    port: 53
    protocol: TCP
  - name: dns
    port: 53
    protocol: UDP
  # here, I was expecting the DNS metrics port 9153,
  # but is not present like other K8s distros 

 Now, my query is, how do I fetch Prometheus metrics of kube-dns in GKE? 
Please help on this, thanks

[garisingh]

The dns-masq agent exposes metrics on port 10054 and kube-dns metrics are on port 10055.