Can't display custom metrics in dashboard

[totodede]

Hello!

I'm currently doing a study project where I'm assigned to do Monitoring.

I've followed this tutorial to help begin and build my first dashboard.

But when I'm trying to visualize information in the dashboard, I never see the information working.

The "no data" pic :

My custom metric :

The kind of logs I'm trying to measure :

I hope I provided enough information, and sorry if I didn't, I'll stay in touch for any further information if needed.

Thank you sooo much for your help!!

[DamianS]

Hello,
Thank's for writing here. Basically, your screenshots presents [1]log-based metrics , not [2]custom metrics or user-defined metrics ( those names can together) . Your configuration is partially okay, however, under the filter field, you should specify filtr for logs [3] . So you must combine [4]query and based on that query create log-based metric and alert. Examples can be found at Google's documentation. Apart from that, if you are trying monitor https latency for cloud run, wouldn't be better to use [6]metric provided by Google? 

[1] https://cloud.google.com/logging/docs/logs-based-metrics
[2] https://cloud.google.com/monitoring/custom-metrics#:~:text=User%2Ddefined%20metrics%20are%20sometime....

[3]
[4] https://cloud.google.com/logging/docs/view/logging-query-language
[5] https://cloud.google.com/logging/docs/view/query-library
[6]

cheers.
DamianS

[DamianS]

Additionally,

I've made simple cloud run revision,  and provided few queries for cloud run, [1]httpsRequest status code, [2]latency less than value_in_sec

[1]

resource.type = "cloud_run_revision" AND
log_name="projects/product-pages-prod/logs/run.googleapis.com%2Frequests" AND
httpRequest.status="200"

 [2]

resource.type = "cloud_run_revision" AND
log_name="projects/product-pages-prod/logs/run.googleapis.com%2Frequests" AND
httpRequest.latency > 0.005744282s

[totodede]

Hello DamianS, thank you so much for your input.

I was trying to get the latency to check if I could retrieve data before trying to measure different key information.

Unfortunately I wasn't able to make anything work despite all your help, there may be something I missed or didn't understand. Anyway I've moved on from this objective and I'm trying a different scenario.

From these logs :

I'm trying to find an MQL query that would build a line chart

I'd like this query to have in axis X the time with an hour or a minute interval and in axis Y, the number of 403 status codes retrieved for the X hour.

I have checked the documentation but I'm struggling a lot building this query...

https://cloud.google.com/monitoring/mql/examples?_ga=2.122959883.-1251670378.1673862564&hl=en

I also asked our ChatGPT friend

fetch gce_instance
| metric 'logging.googleapis.com/operation'
| filter httpRequest.status = 403
| group_by [TIMESTAMP_TRUNC(timestamp, HOUR)]
| count()

 But they couldn't find the 'logging.googleapis.com/operation' metric which I don't really understand...

I hope you can help me again.

Thank you so much in advance.

[DamianS]

Hi,

I didn't fund any metric like logging.googleapis.com/operation. IMO if you trying to fetch something from gce_instance, maybe ops agent is mandatory to obtain this metric. I'm guessing that tbh. What you can do or ( what I'm suggest to do ) is install ops agent at any VM and check whether this metric will be available or not. Additionally, you can create log-based metric with filter request 403 and fetch data from that metric ( dunno if this will work 😄 ) 
PS1: Don't trust chatGPT 😄 
PS2: If my advices will not work, try to create issue at issue tracker. Maybe Google's support will have more info about that. 

cheers,
DamianS