Does vertex ai provides any model monitoring dashboard to visualize model performance

[trk]

Hello GCP Community,

I'm working with Vertex AI and want to enhance my understanding of model performance through visualization. Are there built-in dashboards in Vertex AI Model Monitoring that offer insights into model performance metrics, predictions, and feature distributions?

I'm also interested in any integrations or connectors for tools like Looker or Grafana for model monitoring. If you have experiences or recommendations for setting up such integrations, I'd love to hear them.

My goal is to gain a comprehensive view of model performance, including prediction accuracy, distribution of predictions, and feature behavior over time.

Any insights or advice you can provide would be incredibly helpful.

[Poala_Tenorio]

Vertex AI, Google Cloud's managed machine learning (ML) platform, provides monitoring and visualization tools to help you gain insights into model performance.

Vertex AI provides built-in dashboards that offer insights into model performance metrics, predictions, and feature distributions. These dashboards can help you monitor and analyze various aspects of your deployed models. You can visualize metrics such as prediction drift, data drift, and other performance indicators. This helps you understand how well your model is performing over time and identify potential issues.

While there might not be direct integrations with Looker or Grafana, you can still achieve the desired integrations through the following approaches:

Looker Integration:

You can use Looker to connect to BigQuery, where your monitoring data might be stored. By querying the necessary tables/views, you can create custom Looker dashboards to visualize and analyze model performance.

Grafana Integration:

If your monitoring data is stored in a compatible data store, you can use Grafana to create custom dashboards. Grafana supports various data sources, and if your monitoring data is accessible, you can set up Grafana to query and visualize the data.

[trk]

Thank you for the above response . This really saves lot of time  wrt integrations. I see above model monitoring metrics are only possible for tabular models both autoML & custom trained. What about other models like text classification , image classification  in both automl & custom trained ? And also how to deal with monitoring LLm models  ?