JSON payload modify- parameter all values to be in one-liner

[ravivish83]

We get the part of JSON looks like below as one parameter contains different lines ,which causes issue at Apigee side to forward the same to target endpoint

{"error_message":"Received non success response code","error_details":{"path":"\/api\/v1\/name","error":"Bad Request","message":"","timestamp":"2024-01-29T12:01:27.048+00:00","status":400},"error_code":"org.mozilla.javascript.Undefined@

"passedSummary" : "(alertID: 1234567 ) ***********************FootPrintSummary_Start***********************

WorkflowStartTimestamp : 19/01/2024 18:02:19

WorkflowEndTimestamp : 19/01/2024 18:03:05

WorkflowExecutionTime : 00:0:46

WorkFlowPattern: DB_ERROR

WorkflowOutcome : Escalated

WorkflowOutcomeReason : Escalated

Object : abcd12345_iq

Object Class : NA

HostName : 1234abcd.net

ServiceStream : sampleservice

AutomationType : Runbook

***********************FootPrintSummary_End***********************

Thanks and Regards,

Team

(Do not reply - This is an auto-generated mail)Escalated to team" ,

"passedResultsURL" : "Escalated || SUPPORT

"

so, is there a way to keep all different lines to one line for the parameter value like below, 

"passedSummary" : "(alertID: 1234567 ) ***********************FootPrintSummary_Start***********************WorkflowStartTimestamp : 19/01/2024 18:02:19WorkflowEndTimestamp : 19/01/2024 18:03:05WorkflowExecutionTime : 00:0:46WorkFlowPattern: DB_ERRORWorkflowOutcome : EscalatedWorkflowOutcomeReason : EscalatedObject : abcd12345_iqObject Class : NAHostName : 1234abcd.netServiceStream : sampleserviceAutomationType : Runbook***********************FootPrintSummary_End***********************Thanks and Regards,Team(Do not reply - This is an auto-generated mail)Escalated to team" ,"passedResultsURL" : "Escalated || SUPPORT"

[dchiesa1]

What generates the payload that you're showing there?

Does your Apigee receive that JSON from some external systems? Or does your Apigee proxy create that thing?

If you are creating the JSON payload within the proxy, something like this:

<AssignMessage name='AM-assign-json'>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <AssignVariable>
    <Name>json-payload</Name>
    <Template>{
  "error_message" : "Received non-success response code", 
  "status" : {response.status.code},
  "error_code" : "{variable-containing-newlines}"
}
</Template>
  </AssignVariable>
</AssignMessage>

...what that does is...the content within variable-containing-newlines is just string-replaced into that JSON template.  With newlines in that content, the result will be invalid JSON.  Maybe something like what you showed. 

If that is the case you're doing, you will want to use  the escapeJSON function  within the Message template.  It looks like this:

<AssignMessage name='AM-assign-json'>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <AssignVariable>
    <Name>json-payload</Name>
    <Template>{
  "error_message" : "Received non-success response code", 
  "status" : {response.status.code},
  "error_code" : "{escapeJSON(variable-containing-newlines)}"
}
</Template>
  </AssignVariable>
</AssignMessage>

This will escape newlines so that the generated json is valid.

[ravivish83]

Thanks for the reply.

what generates the payload that you're showing there?

= {"error_message":"Received.......... is the error message which received on Apigee when external system sends JSON payload to the Apigee.

Does your Apigee receive that JSON from some external systems?

=Yes, external system sends JSON

the actual payload to Apigee is like

{

"passedSummary" : "(alertID: 1234567 ) ***********************FootPrintSummary_Start***********************

WorkflowStartTimestamp : 19/01/2024 18:02:19

WorkflowEndTimestamp : 19/01/2024 18:03:05

WorkflowExecutionTime : 00:0:46

WorkFlowPattern: DB_ERROR

WorkflowOutcome : Escalated

WorkflowOutcomeReason : Escalated

Object : abcd12345_iq

Object Class : NA

HostName : 1234abcd.net

ServiceStream : sampleservice

AutomationType : Runbook

***********************FootPrintSummary_End***********************

Thanks and Regards,

Team

(Do not reply - This is an auto-generated mail)Escalated to team" ,

"passedResultsURL" : "Escalated || SUPPORT

"}

It can be resolved if the above payload formats to

{"passedSummary" : "(alertID: 1234567 ) ***********************FootPrintSummary_Start***********************WorkflowStartTimestamp : 19/01/2024 18:02:19WorkflowEndTimestamp : 19/01/2024 18:03:05WorkflowExecutionTime : 00:0:46WorkFlowPattern: DB_ERRORWorkflowOutcome : EscalatedWorkflowOutcomeReason : EscalatedObject : abcd12345_iqObject Class : NAHostName : 1234abcd.netServiceStream : sampleserviceAutomationType : Runbook***********************FootPrintSummary_End***********************Thanks and Regards,Team(Do not reply - This is an auto-generated mail)Escalated to team" ,"passedResultsURL" : "Escalated || SUPPORT"}

[dchiesa1]

ok - if you're saying the external system is sending you JSON with unescaped newlines within it, then.... the external system is mis-formatting the JSON.  The external system is in error.  You need to tell that external system, to fix that bug. 

"Fixing" that issue within Apigee, is the wrong thing to do.  You could do it, but the solution will be brittle and unreliable. 

[ravivish83]

Hi,

yes , that fix should be at external system but this will be taking longer than expected and meanwhile I have to do the workaround for the new end to end solution validation.

Hence, can you suggest the optimal approach at Apigee side. 

Thanks,

[dchiesa1]

ya, I guess if I were doing this I would write some JS to fixup that broken payload, using RegExp or other string matching logic. I'm sure it would be brittle and hard to maintain, though.

The development process will be iterative. If you follow the process of embedding the JS into a proxy as a JS callout, and then deploying the proxy, waiting for that to finish, then sending in a request.... The Edit-Build-Test-Debug loop will be very tedious. You will need to wait 60s or more for deployment to succeed. Lots of steps. Not good. Ideally you would be able to run and test the JS you are building, outside the Apigee environment. Like, from your terminal.

As you know the Apigee JS callout relies on the Mozilla Rhino JS engine. It's not v8, and it's not node or deno. One implication of that is that you cannot include arbitrary npm modules, and you cannot rely on arrow functions or spread syntax, promises, and more things that have been added into EcmaScript over time. A second implication is that ... you cannot use node to run/test your JS!

You need a way to run the JS, directly in the Rhino engine.

So, to aid in development of JS callouts for Apigee, when the JavaScript logic is not completely straightforward, such as in this case, it's good to get a stand-alone Rhino environment, where you can run Rhino on your own workstation, from the command line. This allows you to write JS, then run it directly in Rhino, so the Edit-Build-Test-Debug loop is pretty efficient. For each change you make, You do not have to embed the JS into a proxy then deploy the proxy, wait for that to finish, then send in a request. You can do it all from a terminal.

Here's what I did to set up my environment.

1. Download Mozilla Rhino from the github repo. There is a releases page. Download the zip from there. Per the Apigee documentation, Apigee X relies on v1.7.13 of Rhino, so you might want to download THAT release, as opposed to a later one. Not sure which version of Rhino is used by OPDK or Apigee Edge.
   To use  v1.7.13 from your terminal, you need to download rhino-1.7.13.jar and rhino-runtime-1.7.13.jar

2. you also need Java, probably Java8 or Java11.  I haven't tried running Rhino in later JVMs.

   $ java -version 
   openjdk version "1.8.0_382"
   OpenJDK Runtime Environment (build 1.8.0_382-bre_2023_09_05_21_07-b00)
   OpenJDK 64-Bit Server VM (build 25.382-b00, mixed mode)
   

3. write your JS

4. run/test it like this:

   $ java -cp rhino-runtime-1.7.13.jar:rhino-1.7.13.jar \
         org.mozilla.javascript.tools.shell.Main -f ./my-source-file.js
   

You can use print() to see output.  There is also debugger support in Rhino, but I don't know how to use that.

This has been a huge time-saver for me, as I build logic in JS that runs in Rhino. 

One tip

• I found that if I define a local variable named context, in my script.... Rhino will choke.  I forget the exact problem, but the script will fail to run. That was true in v1.7.11, not sure if still true in v1.7.13.  Anyway, just avoid defining a variable of that name.  This presents a minor challenge if you want to mock the message context object that is present in Apigee, in your script.  In Apigee, that thing is available as variable context. To avoid this, I just use a different variable name for my mock - something like ctx.  And then I have to replace names when I deploy the script into Apigee.