ExtractVariables unexpected behaviour

[nmarkevich]

Hi there,

I am assessing how ExtractVariables policy works on Apigee.

I have noticed that extracting path variables working as not expected when comma (,) or semicolon (;) or equal (=) are passed.

Consider following example:

ExtractVariables policy:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="false" enabled="true" name="default-Extract-Variables">
    <DisplayName>Default Extract Variables</DisplayName>
    <Properties/>
    <URIPath>
        <Pattern ignoreCase="true">/{pathparam}/rest</Pattern>
    </URIPath>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <Source clearPayload="false">request</Source>
    <VariablePrefix>apigee</VariablePrefix>
</ExtractVariables>

 A simple request:

curl -X GET \
  'https://BASE_URL/beforecomma,aftercomma/rest' \
  --header 'Accept: */*' \
  --header 'User-Agent: Thunder Client (https://www.thunderclient.com)'

Expected result: value of apigee.pathparam is beforecomma,aftercomma. Full path segment extracted.

Actual result: value of apigee.pathparam is beforecomma.

According to the rfc3986,

...the semicolon (";") and equals ("=") reserved characters are
   often used to delimit parameters and parameter values applicable to
   that segment.  The comma (",") reserved character is often used for
   similar purposes...

But as I understand from the Apigee's documentation Apigee should extract path segment fully, not segment's parameters.

And when you see the following pattern, you expect to get all content between "/" and "/rest", right?

<Pattern ignoreCase="true">/{pathparam}/rest</Pattern>

[dchiesa1]

I dunno, I guess ExtractVariables doesn't do what you expect when the URL includes commas. 

If you want to extract the segments that include commas you can use JavaScript to do it.  OR avoid using commas in the URLs.

Good luck!

[nmarkevich]

I dont want to extract commas, I thought that using pattern like /path/{var}/path will force Apigee to extract all content in between...

So this behaviour can be considered normal? I dont understand why 🙂

[nmarkevich]

I think this problem is not only about using commas in path. So I put API Security tag not by accident.

According to the apigee docs, I can use Javascript to validate parameters in order to mitigate Injections.

Lets imagine that someone wants to validate path parameter.

Firstly, this path parameter will be extracted via ExtractVariables policy and assigned to some variable. - this is the most obvious!

Secondly, this variable will be validated in a way like this

if (parseInt(path_var)) {
    throw("Mailformed path parameter")
}

In that case an attacker can pass a path which will pass this check, for ex. 42,'; SELECT SLEEP(10) --

But Apigee developer be sure that his API validates everything and will not allow this 🙂

[dchiesa1]

Yes I agree with your statements. You're right. I've filed a bug (internal ref b/242078304) requesting that we add this capability to ExtractVariables.

It won't be possible to change the default behavior of ExtractVariables, because some people may be depending on it, as it works today. But we could add a new flag, maybe named includePathSegmentParameters :

  <URIPath>
    <Pattern includePathSegmentParameters='true'>/{seg1}/{seg2}/baz</Pattern>
  </URIPath>

Note: This is not currently supported. I am saying we COULD add a flag like that.

I would propose that with a path like /foo/bar;bam/baz , then with this flag you would get seg2 = 'bar;bam'.

[nmarkevich]

Thanks for that 👍

Looking forward to a solution

[hmohammad]

Hello @dchiesa1 , do we have any  update on the defect you had raised - b/242078304.

Can we track it anywhere?

[dchiesa1]

checking

Update- no, the change has not yet been approved and merged. The team is working on it.

[verkiy]

You might need to adjust the policy configuration to properly handle special characters like commas in the URI path.

[hmohammad]

Any specific configuration change? any example would be appriciated.