I have a requirement where i have to restrict users based on the users account level instead of proxy level. We have around 40000-50000 users. Is it possible
Yes, you can. At the moment, the way to do it is sort of roundabout. I'll explain.
The Quota (Rate limit) policy in Apigee accepts an identifier, which is used to identify the "buckets" into which Apigee aggregates the call counts. For example if you use the client_id as the Identifier in a Quota policy, then there's a distinct "bucket" for each CLIENT, which in Apigee parlance is a single Application , produced by a single developer. Imagine a developer builds an app, embeds Apigee app credentials within it, an then distributes the app to 100 people. Those people run the app on their phones. Each instance of the app uses the same client_id. If there is a rate limit enforced using the client_id as the identifier, then Apigee bundles up the calls for all 100 people into one bucket. One person might use 90% of the allotted quota (whatever it is; 1000 calls per minute, 1000 calls per hour, whatever), or ... two people might use 100% of the quota and after that, Apigee would reject (429 rate limit) calls from any of the 98 other instances of that app. Sometimes this is what you want.
You could also use developer_id as the Quota identifier. In that case, Apigee would bundle calls for all apps by a single developer into a single counting bucket. If a single developer builds 5 apps and then distributes each one to 100 people, there will be 500 instances of the app running, and Apigee would aggregate calls for ANY of those app instances into a single quota bucket.
And you can enforce multi-level quotas, so you could enforce a rate limit per developer AND per app. You'd use two quota policies to do that, each with a different Identifier, and maybe (probably) with a different numeric rate limit. Eg 10000 requests per minute per developer, and 1000 requests per minute per app.
It sounds to me that you want to use a user_id for the quota bucket identifier. No problem. As long as you can distinguish inbound calls by user, then you have a user identifier and you can configure that as the identifier in the Quota policy. Apigee will do the right thing. Many times Apigee won't have access to an identifier that securely identifies the USER which is calling in. But if your access token includes some sort of user id, then that will work to distinguish users from each other, and it's a good candidate for the Identifier in the Quota policy.
There is one twist - implicitly Apigee adds the proxy name to the Quota bucket identifier. So if you specify "user_id" , then the actual identifier for the quota counting bucket is not simply the user id, it's the proxy name concatenated to the user_id. The description I provided above, regarding how the counts are aggregated across app instances, is true, but it's done on a per-proxy basis. So if proxyA uses user_id as a bucket identifier in the Quota policy, and proxyB uses user_id as a bucket identifier in its Quota policy, there will be two distinct buckets.
Some people don't want that. Currently there is no configuration option to "turn off" the implicit concatenation of the proxy name to the quota bucket identifier, although there is a feature request to provide this (internal ref b/149229963). The way to get around it is to create a single common Quota policy, and attach it to an internal proxy. Then from each proxy that should share a quota bucket, call that internal proxy (via ServiceCallout with LocalProxyConnection) to enforce the quota. The proxyname still gets concatenated to the bucket identifier, but because you're using a single common proxy to enforce quota, it's always the same proxyname, so you get the same effective quota bucket identifier across all your "External" proxies.
Does this make sense?
Thank you @dchiesa1 for your reply
Hi @dchiesa1 Thanks for your reply. Can you pls tell what is the maximum number of hits a user/ Developer can make per minute if we apply quota policy .
@GRaj wrote:
Can you pls tell what is the maximum number of hits a user/ Developer can make per minute if we apply quota policy .
Well that is up to your configuration. You choose that limit. It's part of the configuration of the quota policy. You can make it 1000 per minute, or 10000, or ... you choose.
I have a use case which we want to implement via Apigee. We want to restrict users based on the tier they belong to. We have a backend Jira system to which our users will be hitting. In our current project we have developed a custom script using Node.js. What the script does is, it restricts the users on hourly and Daily basis on the tier they belong to. we have divided users in to 5 categories. 1)General category - where users can hit up to 2000 request/ day and on reaching 15000/day a warning email will be sent to the user that he is nearing the quota and he will be blocked if he crosses 2000 request. 2) Exempted users - This user can do unlimited calls based on their business requirement
similarly, we have 3) Tier-1 4) Tier-2 5) Tier-3 users who will also come under the rate limiting category and each one has different rate limiting set based on hourly and daily basis . On reaching certain limit they will be sent warning email and when they reach the limit set based on their tier they will be blocked.
We have these 2 scripts which run on hourly basis and one run on daily basis. so when the hourly cron run, it checks the blocked users from the hourly block list and unblock them. similarly Daily cron run and it will unblock. The hits will be compared with ELK system based on that user hits will be calculated. Though Jira 9.0 version has rate limiting option but when we are using token-based authorization, Users are saving the tokens in cookie and Jira system unable to identify the users and so rate limiting is not working effectively. so, for that we have introduced the custom script. So, to want to achieve the same via Apigee. Is this possible via Apigee. Then how we can achieve this. How we should design our proxy, and which is the effective policy to achieve this. If we use quota policy, how to configure it. Can we use custom script? Need help on this. Whether we can achieve this with Apigee or not as we want to restrict user based on their user account level but not at proxy level.
You can configure the Apigee Quota policy to enforce a dynamic and varying quota, on a per user basis. You can accomplish what you want.
There are different "types" of Quota in Apigee. Rolling Window, Calendar, Flexi, and "default". Let's suppose you want a limit of 2000/day, and you want the day to start at midnight. For that you can use the Calendar type, and you can specify a StartTime of midnight (for your local time). That means the counting interval starts at midnight, whether or not the user is making any requests. (There are other Quota types that begin the counting interval, upon receipt of the first request).
ok so the simple case is a static configuration: 2000 requests per day . This looks like the following:
<Quota name="Quota-1" type="calendar">
<StartTime>2021-02-18 10:30:00</StartTime>
<Interval>1</Interval>
<TimeUnit>day</TimeUnit>
<Allow count="2000"/>
</Quota>
And then you would attach that Quota policy in the request preflow of the proxy, it would look like this:
<ProxyEndpoint name='endpoint1'>
<Description>Proxy Endpoint 1</Description>
<HTTPProxyConnection>
<BasePath>/whatever</BasePath>
</HTTPProxyConnection>
<PreFlow name='PreFlow'>
<Request>
<Step>
<Name>Quota-1</Name>
</Step>
</Request>
<Response>
</Response>
</PreFlow>
...
But what you want is a varying allowance. For some people it will be 2000, for others it will be 3000, for others it will be unlimited. I suppose the determination whether a user should be given 2000, 3000, or unlimited will be externally determined - it is not encoded in the Apigee proxy itself, but is determined by some other logic that runs external to the proxy. Call it a microservice. The ratings microservice.
To make this happen you need to configure Apigee to call into the ratings microservice before it tries to enforce the quota. You might run the rater, every hour, or every day. But let's expose the computed ratings data via a service. Then, in the flow, you must use a ServiceCallout policy to retrieve the rating for the current user. But what is the current user? Maybe that is determined by the inbound JWT access token. So first you must verify the JWT, then make the call to the ratings service, then enforce the quota. The flow now look slike this:
<ProxyEndpoint name='endpoint1'>
<Description>Proxy Endpoint 1</Description>
<HTTPProxyConnection>
<BasePath>/whatever</BasePath>
</HTTPProxyConnection>
<PreFlow name='PreFlow'>
<Request>
<Step>
<!-- verify the user and get the inbound token -->
<Name>VerifyJWT-1</Name>
</Step>
<Step>
<!-- call to the ratings service -->
<Name>SC-Ratings-Service</Name>
</Step>
<Step>
<!-- enforce the dynamic quota -->
<Name>Quota-1</Name>
</Step>
</Request>
<Response>
</Response>
</PreFlow>
...
Let's suppose the response from the ratings service is a JSON payload. Depending on the response from the ratings service your quota policy will do different things. So we need a few more changes. We need to extract the "tier" of user from the response, and also the allowed # of calls. So we can use an ExtractVariables policy to do that. Then we have context variables that hold the information we seek. Then, enforce the quota IF and only IF, the user is not in the unlimited tier. So we surround the quota with a Condition. And also, if we do actually enforce the quota, apply the rate limit that the ratings service gave us. The quota policy now looks like this:
<Quota name="Quota-1" type="calendar">
<StartTime>2021-02-18 10:30:00</StartTime>
<Interval>1</Interval>
<TimeUnit>day</TimeUnit>
<Allow countRef='variable_containing_count'/>
</Quota>
and in the flow it is wrapped like this :
<ProxyEndpoint name='endpoint1'>
<Description>Proxy Endpoint 1</Description>
<HTTPProxyConnection>
<BasePath>/whatever</BasePath>
</HTTPProxyConnection>
<PreFlow name='PreFlow'>
<Request>
<Step>
<!-- verify the inbound token, and get the inbound user -->
<Name>VerifyJWT-1</Name>
</Step>
<Step>
<!-- call to the ratings service, to get the rate limit per user -->
<Name>SC-Ratings-Service</Name>
</Step>
<Step>
<!-- extract the response from the ratings service, into variables -->
<Name>EV-Ratings-Data</Name>
</Step>
<Step>
<!-- conditionally apply the dynamic quota using the extracted variables -->
<Condition>user_tier != "unlimited"</Condition>
<Name>Quota-1</Name>
</Step>
</Request>
<Response>
</Response>
</PreFlow>
...
That's the basic structure. Now of course, you might not want to call the ratings service for every inbound call. So maybe you wrap the ServiceCallout with a cache. Basically insert the response into a cache, and check the cache before the ServiceCallout.
And maybe the inbound token is not a JWT; that's ok, Apigee can verify tokens of a different type. The main thing you need there is a way to extract a user identifier from the inbound credential (token) for the call.
Does this make sense? You could do this yourself, though it may take a bit of iteration to sort it out.