Solved: '/PredictionService.Predict' from role 'cloud-llm-... - Page 2

gaieges · 02-25-2025 09:05 AM

Hey folks,

I've been getting this weird message sporadically when using Cloud Text-To-Speech Long Audio Synthesis functionality that indicate system capacity limits. This function has an exponential backoff and is still failing even after 7 attempts

The error message I get is:

400 Request '/PredictionService.Predict' from role 'cloud-llm-api-servo-model-owner' throttled: Service is overloaded (in-flight-requests) go/tr-o. 3: Request '/PredictionService.Predict' from role 'cloud-llm-api-servo-model-owner' throttled: Service is overloaded (in-flight-requests) go/tr-o."

I'm wondering if instead this is hitting quotas on my own account instead. I've looked at the usage graph on my API usage and it doesn't seem like it is even throwing 400s from the backend during the time we had issues. But perhaps that's expected for this metric?

Wondering if anyone has any thoughts as to what's going on here.

gaieges

Figured this out on my own, the issue was that I was using some of the old Journey voices, which, Google subtly has provided less capacity for. In turn it was taking VERY long to run or it was just failing like the above. Switching to the new Chirp3 HD models helped.

View solution in original post