Understanding Logging metrics in Cloud Operations

[Peace]

Hi,

I'm new to GCP and trying to get a better grasp of how to effectively use Cloud Operations for monitoring my resources. More specifically, Custom log-based metrics: How do I go about creating metrics based on specific logs generated by my applications? I'd like to be able to track things like error rates or specific event occurrences.

[DamianS]

Hello @Peace ,

Welcome on Google Cloud Community.

Cloud Operations stack ( formerly Stackdriver ) is a great cloud-native tool for monitoring and logging purposes. Basically, creation of log-based metrics and log-based alerts is simple. You must:

1. Decide, on which level you want aggregate logs ( project, folder, organization )
2. Decide whether you need trigger something via matched logs or not ( if yes, PubSub logs-bucket, if not, Cloud Storage logs-storage bucket)
4. This will be more expensive, however, if you need analyze logs via SQL queries in a later time, BigQuery should be also used
5. Combine query, which will search matched logs. You can do it at your own, or use predefined one.
6. Create log-based metric.
7. Create log-based alert with condition, notification channel (slack, email, PagerDuty service).  

Following documentation should help you with that, if not, I'll be more than happy to help you 🙂

[1] How to create query 
https://cloud.google.com/logging/docs/view/building-queries
Samples: https://cloud.google.com/logging/docs/view/query-library

[2] How to create log-based metrics
https://cloud.google.com/logging/docs/logs-based-metrics/counter-metrics
https://medium.com/globant/how-to-create-a-custom-log-based-metric-and-alert-in-gcp-3d08e5d6b7af

[3]. How to create log-based alerts
https://cloud.google.com/logging/docs/alerting/log-based-alerts

[4]. Notification channels
https://cloud.google.com/monitoring/support/notification-options

cheers,
DamianS