Unable to extract a entire path param when the URI is ending with equal sign(=)

[amar333n]

I have a get operation, below is the URL used to invoke the API

{{HOST}}/v1/products/123121212=?q1=53&q2=43

When I use the extract variable policy to extract the path param, = sign is not populating in the variable.

Below is the content of the extract variable policy.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="false" enabled="true" name="Extract-Variables">
    <DisplayName>Extract Variables</DisplayName>
    <Properties/>
    <URIPath>
        <Pattern ignoreCase="true">/{productid}</Pattern>
    </URIPath>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables>

The Value for the variable productid is 123121212.

Did any one faced this issue, please help me to resolve this issue.

Thanks in Advance

[ssvaidyanathan · Accepted solution]

Hi @Amar N

You will need to URL encode for such cases, thats the common practice when you are tying to send special characters in the URI especially for Path Param.

So in your case, it will be

{{HOST}}/v1/products/123121212%3D?q1=53&q2=43

Once its extracted, the value for productid will be 123121212%3D

You can use JS policy to decode that

decodeURIComponent(context.getVariable("productid"))

Let me know if it worked

[amar333n]

@Sai Saran Vaidyanathan

Thank you, it is working now

[ssvaidyanathan]

Awesome.. Glad it did !

[shrenikkumar-s]

This is definitely helpful. However, what if the consumer sends them unencoded, what are our options in that case?
We've a workaround of using pathsuffix and then removing the resourcepath from it to extract the path parameter (like shown below), is there any other thought/way to do this?

var pathsuffix = "/ldap/user/groupId=123456789,ou=Groups,ou=Customers,dc=google,dc=com"; // /ldap/user/ part is static and remaining can be anything

var pp = pathsuffix .replace("/ldap/user/", "");

@ssvaidyanathan 

@amar333n 

@Peeyush_Singhai 

[ssvaidyanathan]

not its always supposed to be URL encoded. You can find it from the RFC spec too

[dchiesa1]

---

@shrenikkumar-s wrote:

what if the consumer sends them unencoded, what are our options in that case?

---

Sai says the client must encode the equals signs (and probably the commas). I'm not sure that it's so clear. 

Per my understanding of the relevant IETF RFC describing URIs, which is RFC 3986, the equals character and the comma are both reserved characters. According to that spec, reserved characters should be percent-encoded UNLESS they are specifically allowed by the specific URI scheme.

   URI producing applications should percent-encode data octets that
   correspond to characters in the reserved set unless these characters
   are specifically allowed by the URI scheme to represent data in that
   component. 

When RFC 3986 refers to "what is allowed by the URI scheme", in the case you are considering, that scheme is HTTP or HTTPS. And URIs for those schemes are defined in RFC 7230, which.... cites RFC 3986 for what it allows. It feels sort of circular.

By my reading, RFC7230 does not specifically say "in HTTP/s URIs you can include unencoded equals and comma characters." but it also does not specifically say "in HTTP/s URIs, you must encode delimiters like equals and comma."

So it's sort of a grey area. 

Requiring your clients to URI-encode the path segments will allow it to "just work" but that might mean updating clients. If you can't do that then you need to modify your proxies as you described - to do the manual parsing. 

There is an outstanding feature request asking for support within ExtractVariables for extracting path segments when there are delimiters like equals signs.  internal reference: b/242078304 .  It has not yet been approved or prioritized for release. You can ask your TAM to escalate that ticket if you wish. No guarantees that it will be released on any particular schedule.

See also, this community post.

[shrenikkumar-s]

Thanks & appreciate your comments @ssvaidyanathan & @dchiesa1 - like you suggested we'll work with our consumers to see what we want/can do. It's good that we've some solution in both the cases 🙂