Issues with DataCapture Policy for capturing API Path

[harishav]

Hi Team, 

We are relying on the apigee metrics api to get the metrics data from apigee and display on our portal. One of the usecase we have is to show the error rate by API, and we have some APIs with variable path. Example: GET '/v1/report/5345632' 

Expectation would be to report the error rate for all the get v1 report calls under one bucket, but by default apigee will report them as different api calls due to variable report id in the path. 

To overcome this issue, we have started using DataCapture policy to report these calls with one generic api path. And the policy code is shared below. This was working fine but recently we started seeing that most of the times, the custom dimension captured is reported as '(not set)'. 

<DataCapture name="DataCapturePolicy" continueOnError="false" enabled="true">
<DisplayName>DataCapturePolicy</DisplayName>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
<Capture>
  <Collect ref="current.flow.name"/>
  <DataCollector>dc_proxy_path_collector</DataCollector>
</Capture>
</DataCapture>

And the flow name is set as below: 

<Flow name="v1/report/{reportId}">
<Description>Get Job Status</Description>
<Request>
<Step>
<Name>DataCapturePolicy</Name>
</Step>
</Request>
<Response/>
<Condition>(proxy.pathsuffix MatchesPath "/v1/report/*") and (request.verb = "GET")</Condition>
</Flow>

Is this the right approach to handle this kind of usecase, if yes, can you suggest what might be going wrong. 

We are using Apigee X. 

Thanks in Advance! 

[dchiesa1]

---

@harishav wrote:

his was working fine but recently we started seeing that most of the times, the custom dimension captured is reported as '(not set)'. 

---

This may be obvious, but let me say it anyway: it will be reported as "(not set)" is if... it is not set by the proxy. The question is, how would it be possible for that datacollector to not be set. 

The dc_proxy_path_collector gets set in the one Conditional flow You showed , the one in which the DataCapture policy is atached.  Is it possible the proxy is handling calls for which that Conditional Flow does not execute?  In that case, I would expect to see "(not set)". 

To be more specific: In the Condition for that Flow, the pathsuffix pattern you are matching on is ""/v1/report/*" and the verb is GET. What happens if you send in a POST? What happens if you send in a GET with a path that does not match?  In those cases your flow will not execute.  And so I would expect your DataCapture policy will not execute, and therefore the dc_proxy_path_collector will be "(not set)". 

Is that what's happening?

[harishav]

Hi @dchiesa1,

Thanks for the response. I got the point you are trying to make,  all the data points should match for dc_proxy_path_collector to capture the name. And i did verified on those lines on the metrics data. Let me put the findings i had around this. 

Below is the snapshot of the metrics data, 

So, here i have drill down basis the proxy basepath, path suffix and request verb. All are aligned, still few of them has 'dc_proxy_path_collector' set to (not set). Below are the numbers for each of them, 

And below is the flow config for this, 

<Flow name="v2/Order/PlaceOrder">
<Description>Place Order</Description>
<Request>
<Step>
<Name>DataCapturePolicy</Name>
</Step>
</Request>
<Response/>
<Condition>(proxy.pathsuffix MatchesPath "/Order/PlaceOrder") and (request.verb = "POST")</Condition>
</Flow>

Please suggest what could be the issue with this setup. 

Thanks,

Harish 

[dchiesa1]

Are there any incoming API calls that are received by, not not explicitly handled. by, your API proxy? 

For example does your API Proxy receive OPTIONS calls, as for CORS support?  Or are there broken or unauthorized apps that are sending in random queries? 

If you invert your analysis with Apigee analytics, you should be able to gain more insight.  Filter on  dc_proxy_path_collector = "(not set)" and examine the verbs and paths of the requests when that is true.  What does this tell you?

[harishav]

Hi @dchiesa1 / team, 

Can anyone please review the comment above and suggest what could be going wrong. 

Thanks,

Harish 

[madhurranjan]

Can you open a support ticket?