Latency issues in proxy request flow

muq_ash · 09-06-2023 10:51 PM

Hi,

I am accepting base64 encoded image in JSON payload. The payload size is 4.5 MB. However, the request is taking around a minute to be processed by Apigee. If the same request is sent to the target endpoint directly, the request is processed within milliseconds. Please refer to the screenshot below for details and advice how to mitigate the latency issues. Thanks.

Request format:
["base64encoded-image"]

API-Evangelist

Did you enable streaming?Try enabling and see if it helps..

https://cloud.google.com/apigee/docs/api-platform/develop/enabling-streaming

muq_ash

I tried but it doesn't make any difference:

API-Evangelist

Why there is latency at Proxy Request Flow started? This seems to be an indication of a resource contention issue on Message Processor. Is it on-prem or Apigee X(Hope you are playing in trial version)? May be speak with support to gather some more information. AFAIK this is due to latency either at network/ MP overloaded on resources..

Outside that if you really want to go further we may need to troubleshoot further by gathering some more data like thread dump,heap dump,disk utilization,memory,cpu.

In short if you have a better setup with MP this should be not happening for <5MB.

muq_ash

Yes I tried, it didn't make any difference

dknezic

Just to check, how are you sending the data to Apigee and target directly? and how are you measuring the timing to the target directly? 4.5MB in milliseconds sounds a bit suspiciously too quick.

muq_ash

Request to Apigee is sent through Postman. However, by sending data directly to target I mean sending curl request to the target endpoint from the Apigee nodes which takes around 4-5 seconds (4000-5000 milliseconds).

dchiesa1

If you have access to the Apigee nodes, I guess that means you are using OPDK. You manage your own installation.

Possibly there is network contention or your MPs are over-burdened and are experiencing IO or cpu contention.

You can diagnose this by checking your MPs for i/o and cpu and networking.

Maybe see this set of suggestions. https://www.site24x7.com/learn/linux/troubleshoot-high-io-wait.html