High response time (and timeouts) connecting on proxy for 3 days in a row

[#1]

Hi,

I'll like to ask whether someone else is suffering from high response times (more than a sec) or even timeouts when using APIGee proxies.

We're using a free-tier instance and this has been a recurring problem since Friday 28th July. Sometimes it got fixed, sometimes the high response times starts occurring again.

Find below a simple benchmark during one of those high-response-cycles done with Apache Benchmak (ab):

ab -n 1000 -c 10 http://podsystem-prod.apigee.net/apps/ping?apikey=XXXXXXXX
This is ApacheBench, Version 2.3 <$Revision: 1706008 {replace0}gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking podsystem-prod.apigee.net (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software:        Apigee
Server Hostname:        podsystem-prod.apigee.net
Server Port:            80

Document Path:          /apps/ping?apikey=XXXXXXXXXXXXXXXXXX
Document Length:        4 bytes

Concurrency Level:      10
Time taken for tests:   289.925 seconds
Complete requests:      1000
Failed requests:        2
   (Connect: 0, Receive: 0, Length: 2, Exceptions: 0)
Non-2xx responses:      2
Total transferred:      216139 bytes
HTML transferred:       4226 bytes
Requests per second:    3.45 [#/sec] (mean)
Time per request:       2899.245 [ms] (mean)
Time per request:       289.925 [ms] (mean, across all concurrent requests)
Transfer rate:          0.73 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:      132  278 1163.7    168   19582
Processing:   241 2583 3783.4    308   16024
Waiting:      240 2582 3783.7    307   16024
Total:        393 2860 4159.6    484   35346

Percentage of the requests served within a certain time (ms)
  50%    484
  66%    645
  75%   7225
  80%   7968
  90%   8327
  95%   8512
  98%  15706
  99%  16808
 100%  35346 (longest request)

As an additional evidence, is it worth showing what our APM tool (newrelic) has been showing regarding the availability of the APIGee proxy. A simple "Ping" monitoring operation that just queries a method of our API through the APIGee proxy has failed 154 times in 3 days.

[anilsr · Accepted solution]

@Felix Ontanon

Free-tier instances are not designed for performance testing / to run production APIs. They are shared pods sharing with thousands of other similar instances. We do not guarantee any SLA's on free pod. They are only for feature evaluation. Trial pods should not be used for production traffic / run any performance evaluations.

Apigee handles billions of API calls every day with 99.999% SLA. Just FYI, 60% of Top World Global Retailers are Apigee Customers & Their API traffic is powered by Apigee. We handle thousands of TPS, Billions of API calls every day on cloud including events like Black Friday. If you are evaluating Apigee, I suggest not to use free instance to understand the performance / scale. Please get in touch with Apigee sales for a demo or PoC.