Re: Regarding my “online_prediction_requests_per_b...

BlueFunny · 09-10-2024 08:25 PM

{
  "error": {
    "code": 429,
    "message": "Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: anthropic-claude-3-5-sonnet. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai.",
    "status": "RESOURCE_EXHAUSTED"
  }
}

I encountered the above problem when trying to use the model `claude-3-5-20240620` and I confirmed that my credits are not fully utilized yet
I've seen some similar posts about this and the problem I'm currently experiencing is almost identical to theirs, my `online_prediction_requests_per_base_model` limit has also gone to `0` and can't be adjusted, can you help me readjust this limit please?
https://www.googlecloudcommunity.com/gc/AI-ML/Receiving-quota-error-when-trying-to-use-bison-chat-mo...

minusahn

having same issue. cannot change quota from 0 even though i have valid form of payment.
activating full account did not change anything.

https://www.googlecloudcommunity.com/gc/AI-ML/RESOURCE-EXHAUSTED-Anthropic-vertex-quota-error/m-p/80...

In the reply from the link, community manager mentioned that free trial customers can use the anthropic models if they have a form of payment.

BlueFunny

Yeah, I've added a payment method and verified the chargeback, but my limit is still locked

AndrewB

I have asked the team to investigate this. In the meantime, I'll reach out to you both over pm so that we can get this fixed on your projects.

caseDay

Same problem. I’m on a paid account, but even though I’ve already paid, I can’t adjust the “online_prediction_requests_per_base_model” quota for claude-3-5-20240620. The system and support team told me to contact Sales, but it seems like they only deal with big companies and don’t respond to smaller users. Reached out but got nothing back.

anderson579

Same issue here, could you please PM me ?

drwbns

Having the same issue as well. Plenty of credits in my account. Have tried setting different regions as well with no luck. I read us-east5 and europe-west1 are the only available regions with Claude sonnet 3.5 but neither seem to work.

zrr

Same issue here, even though I stop Vertex API and reactive it, my `online_prediction_requests_per_base_model` limit still `0`

DrinkBeer

I reach out to sales team and they just send me in circles. They had no interested in helping me.

nikzart

Getting the same error code but my quota is sufficient and i have valid payment method attached. Can't use sonnet on vertex ai. I'm too used to google cloud and its going to be a mess to switch to aws bedrock to use the same model.

RyanMax

does AWS bedrock work well？we met the same problem，maybe we need to switch to AWS too

jarodise

I'm experiencing the exact same problem :

API Error: Status Code 429, [{ "error": { "code": 429, "message": "Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: anthropic-claude-3-5-sonnet. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai.", "status": "RESOURCE_EXHAUSTED" } } ]

spdustin

Precisely the same issue here, with two quotas, one "unlimited" and the other "0", for the same region/model.(us-east5, anthropic-claude-3-5-sonnet)

RyanMax

same problem. Claude 3.5 quota turned to be 0 yesterday.

samakachi

anthropic.RateLimitError: Error code: 429 - {'error': {'code': 429, 'message': 'Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: anthropic-claude-3-5-sonnet. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai.', 'status': 'RESOURCE_EXHAUSTED'}}

same thing goes here, i just signed up for GCP and activated the credits and attached an active paiement card, but still this i still didn't even to get to do 1 request

kitesgarb

I am facing the same issue here. Getting 429 even after adding a valid payment method (I am on a free trial here though).
@AndrewB Can you help me with this. I am stuck since 1 week now.

BlueFunny

My problem was solved not too long ago, but today the problem has resurfaced, I'm not sure what's going on, I've contacted the community management again, waiting for a reply

Shiki22

Anything??

Kamalddr

Same problem 😭 @AndrewB

Ble11

I can't edit the quota of Vertex AI.

DevinderSinghSk

@AndrewB please PM, facing same issue for my project, 0 limit locked.

Greg0819

Are your issues resolved now? I've encountered the same problem.

Ble11

Unresolved issues

ennio

same problem here. been experiencing it for a week now. I have contacted support but they did nothing to help

interfaced

We had issues for months already with low quotas.

But since today the quota was set to 0 and our service is partly unusable. Did anyone get this issue solved? Experiences with AWS Bedrock?

DrinkBeer

It seems google has restricted access to Anthropic models to paying customers with Enhanced Support or better!

interfaced

It is not only a GCP issue. We tried to switch to AWS Bedrock today, but the model is unavailable in all regions on AWS too.

I quickly checked X and other platforms but did not find any information to the issue. Does anyone have insights?

BlueFunny

I think it might be a risk limit to prevent abusive behavior... For some well-known reason.
We tried contacting the community management and they did re-add our quota, but it's very small, it was supposed to be around 30 RPM, but now it's only 5 RPM and it doesn't look like there's any possibility of adjusting it upwards anymore
Looks like that's the answer to that question 😞

RyanMax

our quota is also changed. from 5 to 3😂

maybe need to try 4o or other models

Greg0819

It's already quite good, at least you all can use it. As for me, I've had zero so far.

nukee

I have been experiencing the same issue for about a week now. As this is a known and obvious problem, I would expect much better communication. Since the API and model are listed as “Generally Available,” I would anticipate a stable, production-ready product here. This makes me reconsider using Google Cloud in a production environment.

mohamm-ad

@AndrewBWhat is going on here? This change is breaking applications that trusted GCP / Vertex. It's been > 2 months without clear resolution or explanation.

There are numerous threads on this platform and others:
- https://www.googlecloudcommunity.com/gc/AI-ML/Vertex-Gemini-429-Resource-exhausted/m-p/831269 / https://www.googlecloudcommunity.com/gc/AI-ML/Claude-Sonnet-3-5-V2-quota-error-code-429/m-p/828354 / https://www.googlecloudcommunity.com/gc/AI-ML/Resource-Exhausted-for-anthropic-claude-3-5-sonnet-v2/...
- https://github.com/cg-dot/vertexai-cf-workers/issues/18
- https://stackoverflow.com/questions/78323246/encountered-429-error-quota-exceeded-for-online-predict... / https://stackoverflow.com/questions/79115604/429-resource-exhausted-for-claude-sonnet-3-5-on-vertex-... /

Regarding my “online_prediction_requests_per_base_model” when using the Claude model problem