Viewing OpenTelemetry Resource Attributes in Metrics

[doug53]

I have my GKE service sending OpenTelemetry data to GCP Observability/Monitoring. It's working great, however, I cannot find a way to use the rich OpenTelemetry Resource Attributes to build charts in Metrics explorer.

For example, these are some metrics being sent by the OTEL Collector (this is pulled from collector debug logging):

2024-11-14T19:26:25.814Z info ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.26.0
Resource attributes:
 -> k8s.pod.name: Str(my-api-test-6996648499-lxmw7)
 -> k8s.pod.uid: Str(10f04503-b24b-4f86-a08e-94cf050ff837)
 -> service.name: Str(my-api)
 -> service.version: Str(1.0.3+189-test)

However, I cannot find a way to access these attributes. The closest I see to this is the ability to aggregate on "resource labels":

What I would like to do is graph requests by things like pod name, service version, or any of the other custom resource attributes that are part of my OTEL data.

I see these attributes in the Traces explorer, but not in the Metrics explorer. What am I missing?

Thanks.

[greb]

Hi @doug53,

Welcome to Google Cloud Community!

It seems the concern lies in the discrepancy between OTEL resource attributes and the resource labels supported by Google Cloud Monitoring. While attributes like k8s.pod.name and service.version appear in traces, they aren't automatically available for use in Metrics Explorer.

Suggestions:

1. Use MQL (Monitoring Query Language):
• MQL provides more flexibility than the Metrics Explorer UI. You can use MQL queries to work with OTEL attributes explicitly. Though MQL will be deprecated after July 2025, you may use an alternative the PromQL.
2. Customize OTEL Collector Configuration:
• Configure your OTEL collector config to map resource attributes like k8s.pod.name to GCP-recognized labels.
3. Explore Enhancements:
• Engage with GCP support or file a feature request to add broader support for OTEL resource attributes in Metrics Explorer. This won’t guarantee specific time resolution, but it might help you and the community on your future projects.

I hope the above information is helpful.