Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

MessageLogging policy fails to write logs to cloud logging- logging failure action detection

I have a MessageLogging policy in Post client flow for logging request ,response, policy executed flags variables and so on 
I do not use any javascript policies to set these variables anywhere. for some reason if any of those variables fails to be set ( "developerEmail": "{escapeJSON(developer.email)}",)  the MessageLogging policy Does not throw an error in the debugging session ,does not set any error variables . it also does not write this log to cloud logging.
Is there a way to detect if the message logging action it self  failed (writing those logs not the policy itself because according to the documentation the policy itself succeeds even if logging fails)   for me to take a decision in my proxy accordingly like to log this failure or so ??

dareenhamdy_0-1736699638067.png

https://cloud.google.com/apigee/docs/api-platform/reference/policies/message-logging-policy#fault-va...

 

 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MessageLogging continueOnError="false" enabled="true" name="ML-LogSuccessRequestToCloudLogs">
  <DisplayName>ML-LogSuccessRequestToCloudLogs</DisplayName>
  <CloudLogging>
    <LogName>projects/{projectId}/logs/{apiproxy.name}</LogName>
    <Message contentType="application/json">
      {
        "proxyGeneralInfo": {
            "environmentName": "{environment.name}",
            "apiProxyName": "{apiproxy.name}",
            "apiProxyRevision": "{apiproxy.revision}",
            "apiProductName": "{apiproduct.name}",
            "appName": "{developer.app.name}",
            "clientId": "{client_id}",
            "correlationId": "{messageid}",
            "developerEmail": "{escapeJSON(developer.email)}",
            "callingIPAddress": "{escapeJSON(client.ip)}"
        },
        "requestInfo":{
          "requestUrl": "{request.url}",
          "verb": "{request.verb}",
          "requestReceivedAt": "{client.received.end.time}",
          "requestFormParameters": "{request.formstring}",
          "requestHeaderNames":"{request.headers.names.string}",
          "requestQueryParameters": "{request.querystring}",
          "requestPayload": "{escapeJSON(request.content)}",
          "originalServerResponse":"{escapeJSON(original-server-response)}"
        },
        "responseInfo":{
          "responseStatusCode": "{message.status.code}",
          "responseHeaders":"{message.headers.names.string}",
          "responseSentAt": "{client.sent.end.time}",
          "responsePayload": "{escapeJSON(message.content)}",
          "originalServerResponse":"{escapeJSON(original-server-response)}"
        },
        "proxyExecutedPolicies":{
          "OAuthV2-ValidateAuthToken":{
            "failed":"{oauthV2.OAuthV2-ValidateAuthToken.failed}"
          },
          "SA-ApplySpikeArrest":{
            "failed":"{ratelimit.SA-ApplySpikeArrest.failed}"
          }
        }
      }
    </Message>
    <ResourceType>api</ResourceType>
  </CloudLogging>
  <logLevel>INFO</logLevel>
</MessageLogging>
0 3 334
3 REPLIES 3

Hi @dareenhamdy !

I think that Errorstate and similar concepts are not part of the PostClientFlow.

"for some reason if any of those variables fails to be set ( "developerEmail": "{escapeJSON(developer.email)}",)  the MessageLogging policy Does not throw an error in the debugging session ,does not set any error variables . it also does not write this log to cloud logging"

1. Are you sure that the log is not written because some variables resolution failed?
there are a few other possible reason, like InvalidGoogleCloudLogName, or the service is not set enabled etc.

https://cloud.google.com/apigee/docs/api-platform/reference/policies/message-logging-policy

2. What's the reason behind the variable is not set? Is it because the variable is not defined, or other reason? as far as I remember, undefined/unresolved variable should not cause such issue in messageLogging policy.

3. If it does, then please consider defining default value for your variables:

Default values for variables in message template

example 

<Message>This is a test message. id = {request.header.id:unknown}</Message>

4. "take a decision in my proxy accordingly like to log this failure or so ??"

What action would you like to do in such case?
I'm asking this, because as we are already at postClientFlow, options are quite limited.
Is it an option for you to assemble the log message and/or implement the MessageLogging policy before the postClientFlow?

Best regards,
Marcelo

1) yes am sure of the log name, I am aware of this issue. The logging name is correct and the logging is enabled on the project 

2) no value is not null or undefined that doesnot cause an issue in the proxy but if for example if I missed adding escapeJSON function for a variable this scenario happens.

3) will defining a default value for such mentioned case prevent this ?

4) I might need to log/ detect  that the logging to cloud log it self failed by using another message logging policy in post client flow. Is that detectable because the message logging policy executing doesn’t necessarily always mean the logs are actually written.

Hi @dareenhamdy !
Thank you for your answers.

1. " if for example if I missed adding escapeJSON function for a variable this scenario happens."

Well, logging-related stuff must be designed carefully, and tested very thoroughly.
So I would focus on this part, and I'm convinced that you would decrease the chances of the occurrence of such issues to near 0.

2. Even if you created a second logging policy, what would guarantee that it always works, and  always report the failure of the other logsending / not fails as well?
So would you then implement a 3rd logsending function and so on?
This looks to me a vicious circle, and makes no sense.

You must trust in the implemented solution, which trust you are gping to build up by designing/developing/testing carefully.

3. It's true, that other incidents might occur, but for those I really don't think that you should prepare with such logic. Instead, you can compare built-in apigee metrics with the metrics you get from your cloud-log, or logs provided by your consumers and/or backends, loadbalancers etc.

I also think that there are some solution that provide infrastructure-level logs, that should indicate if any type of exception occurred.

4. Is it an option for you to assemble the log message and/or implement the MessageLogging policy before the postClientFlow?

5. If you still desire to possess the ability to detect any anomaly within proxy runtime, you can do your things before the postClientFlow, using flowCallout policy, and sending the logs to their destination in a commerce rq-rp way. It would be synchron communication, so you will be able to catch any anomaly.
Though I cannot tell you right now how to send logs to the apigee Cloud in this scenario.

I hope I could help,
Best regards,
Marcello