Optimizing Google Cloud Quota Data Collection for Faster Monitoring

[DarwinVinoth]

We typically rely on official Google Cloud APIs to retrieve monitoring data. For quota-related data, we use the Google Cloud Quota APIs.

Currently, Google's Quota details page lists 13K+ available metrics(Only for Compue category alone), making it challenging to fetch all data using a single API request. Our current data collection process takes 40+ minutes, impacting efficiency.

Is there a way to optimize this process—perhaps by leveraging batch APIs or a more efficient method to retrieve quota data in bulk?

Thanks,
Vinoth_GCP.

[mcbsalceda · Accepted solution]

Hi @DarwinVinoth,

You may check this documentation on About Monitoring API quotas.

[mcbsalceda]

Hi @DarwinVinoth, 

You could consider using Google Cloud Monitoring to streamline your data collection process. With Cloud Monitoring, you can set up custom dashboards, create more focused metrics, and leverage metric exports to automate and speed up data retrieval. It also supports batch API requests, which could help reduce the time spent fetching quota data. Plus, you can filter out only the metrics you need, saving both time and resources. It might be a solid way to improve efficiency without having to dig through all the 13K+ metrics.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[DarwinVinoth]

Please share the official documentation for the Batch API request in Cloud Monitoring. 

Thanks. 

[mcbsalceda · Accepted solution]

Hi @DarwinVinoth,

You may check this documentation on About Monitoring API quotas.

[DarwinVinoth]

Can you please share the details about the batch API request? and how can I run that in a code? 

[mcbsalceda · Accepted solution]

Hi @DarwinVinoth,

For more information about batching API requests, see timeSeries.create.