publish message policy in apigeex to send a incoming request content to topic

[raghunathapalle]

@dchiesa1 @kurtkanaskie 

We are trying to use publish message policy in apigeex to send a incoming request content to topic. Is there any pre-requisites for this in apigeex ? also what do we configure in target server in this case? do you have a sample api proxy that we could refer for this use case in apgieex?

thanks in advance!!

raghu

[ricardosm · Accepted solution]

Hello @raghunathapalle! I can help you with this!

There are some pre requisites:

1. You need to have a topic created in Pub/Sub. I suggest using a schema. Schemas will validate the message structure. I will provide examples for json object vs avro schema notation. You can transform a json object into Avro schema using Gemini for example or other tools online. Examples:

- Topic Name: projects/rick-apigeex/topics/orders

- Json Message/object (sample created with Gemini)

{
    "customer_id": 5678,
    "order_date": 1694310800000, 
    "shipping_address": {
        "street": "789 Oak Ave",
        "city": "Springfield",
        "state": "IL",
        "postal_code": "62704",
        "country": "USA"
    },
    "billing_address": {
        "street": "789 Oak Ave",
        "city": "Springfield",
        "state": "IL",
        "postal_code": "62704",
        "country": "USA"
    },
    "payment_method": "credit_card",
    "payment_details": {
        "card_number": "**** **** **** 5678",
        "expiration_date": "03/25",
        "cvv": "***"
    },
    "order_items": [
        {
            "product_id": 345,
            "quantity": 3,
            "unit_price": 12.50
        },
        {
            "product_id": 678,
            "quantity": 1,
            "unit_price": 25.99
        }
    ],
    "order_total": 63.49,
    "order_status": "processing"
}

- AVRO Schema (sample created with Gemini)

{
    "namespace": "com.example.orders", 
    "type": "record",
    "name": "Order",
    "fields": [
      { "name": "customer_id", "type": "int" },
      { "name": "order_date", "type": { "type": "long", "logicalType": "timestamp-millis" } },
      { "name": "shipping_address", "type": {
          "type": "record",
          "name": "Address",
          "fields": [
            { "name": "street", "type": "string" },
            { "name": "city", "type": "string" },
            { "name": "state", "type": "string" },
            { "name": "postal_code", "type": "string" },
            { "name": "country", "type": "string" }
          ]
        } 
      },
      { "name": "billing_address", "type": "Address" }, 
      { "name": "payment_method", "type": "string" },
      { "name": "payment_details", "type": {
          "type": "record",
          "name": "PaymentDetails",
          "fields": [
            { "name": "card_number", "type": "string" },
            { "name": "expiration_date", "type": "string" },
            { "name": "cvv", "type": "string" }
          ]
        } 
      },
      { "name": "order_items", "type": {
          "type": "array",
          "items": {
            "type": "record",
            "name": "OrderItem",
            "fields": [
              { "name": "product_id", "type": "int" },
              { "name": "quantity", "type": "int" },
              { "name": "unit_price", "type": "double" }
            ]
          }
        } 
      },
      { "name": "order_total", "type": "double" },
      { "name": "order_status", "type": "string" }
    ]
  }

2. You'll need a service account with permissions to publish messages in pub/sub. For example, in my case:

• Name: pub-sub-publisher@rick-apigeex.iam.gserviceaccount.com
• Role: Pub/Sub Publisher
• You will need to attach this service account to your Api proxy deployment in Apigee.

To address the question of "what to configure in target server", this depends. If you are creating an API proxy for the sole purpose of

• Receive json payload
• Mediate, transform, create message to publish
• Publishing message to Pub/Sub

You can actually achieve this with a No Target API proxy. Optionally (and I recommend it), you can configure the custom client response from the API Proxy with an Assign message policy and create some conditionals along with RaiseFault policy to control logic, behaviour and handle some errors.

You might also encounter a use case where the message you want to publish into Pub/Sub needs some information that will be responded by the target server. In this case

• You would configure a target server (with a reverse proxy, common approach)
• Request the data
• Extract the data from the response
• Create the message object (json)
• Make the message ingestion into Pub/Sub afterwards.

You can mix this use cases to achieve other ones. For example, you con publish a message into Pub/Sub asynchronously while requesting data from the backend, etc, etc.

I'll attach some example policies I'm using and a proxy image for you to review.

Publish Message Policy:

This is configured in my conditional flow "POST - /orders" in the request:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PublishMessage continueOnError="false" enabled="true" name="PM-publish-new-order">
  <DisplayName>PM-publish-new-order</DisplayName>
  <Source>{request.content}</Source>
  <CloudPubSub>
    <Topic>projects/rick-apigeex/topics/orders</Topic>
  </CloudPubSub>
</PublishMessage>

This is configured in my conditional flow "POST - /orders" in the response:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage continueOnError="false" enabled="true" name="AM-set-response">
  <DisplayName>AM-set-response</DisplayName>
  <Properties/>
  <Set>
    <Payload contentType="application/json">{
    "result": "SUCCESS",
    "description": "order created successfully"
}
    </Payload>
    <StatusCode>201</StatusCode>
    <ReasonPhrase>Created</ReasonPhrase>
  </Set>
  <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
  <AssignTo createNew="false" transport="http" type="response"/>
</AssignMessage>

Something like this: This is a very simple example I created.

I hope this helps and please, if you have some followup questions, let me know! 😄

[shefali]

@ricardosm : could you please explain how to use the same publish message policy to push a message to a topic which is part of another GCP project. (not same as apigee project).
Thanks!