Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Data Collectors, DataCapture & many proxies: capture data from all the proxies: best practices need,

Hi, Community!
I have quite a big number of proxies under my support.
Management would like to collect analytics data
(app names, user names, dates-times & so on)
from all of the proxies & export data to BigQuery.
Exported data is going to be consumed &
processed by some Machine Learning.

Given:

  • many proxies.
  • following Data Collectors: "dc_app_name", "dc_req_email",
    "dc_req_username", "dc_response_header_date" & so on.
  • separate DataCapture policies in each proxy.
  • OR SharedFlow with DataCapture policy.


My question is, what are the best practices for the "Data Collectors" & "DataCapture" policy?
Specifically, from the documentation & my tiny Proof Of Concept, it seems to me that
at first look,

Apigee Hybrid captures data, e.g. emails from all the many proxies
into the single "dc_req_email" resulting that all the emails from all proxies being stored mixed,
"Many proxies - to One data collector (emails)".

However,  in the DataCapture policy  |  Apigee X  |  Google Cloud there is a note:

Show More
  • If you use a Data Collector in multiple policies, the captured data will be overwritten by the last policy that executes.
0 3 898
3 REPLIES 3