Apigee Hybrid 1.7 Cache entry doesn't expire with Date in the past

[Giupo]

Hello, 

we have tried to store access token in cache, but we noticed that cache keys don't expiry.

This is my Assign Message policy to convert epoch time to date (for Populate Cache parameters)

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage continueOnError="false" enabled="true" name="AM_Expiry">
    <Properties/>
    <AssignVariable>
        <Name>expiry</Name>
        <Value>1662456111</Value>
    </AssignVariable>
    <AssignVariable>
        <Name>timeFMT</Name>
        <Value>HH:mm:ss</Value>
    </AssignVariable>
    <AssignVariable>
        <Name>dateFMT</Name>
        <Value>MM-dd-yyyy</Value>
    </AssignVariable>
    <AssignVariable>
        <Name>timeOfDay</Name>
        <Template>{timeFormatUTC(timeFMT,expiry)}</Template>
    </AssignVariable>
    <AssignVariable>
        <Name>expiryDate</Name>
        <Template>{timeFormatUTC(dateFMT,expiry)}</Template>
    </AssignVariable>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</AssignMessage>

For example, in debug session it appears like:

timeOfDay=09:21:51
expiryDate=05-06-2022

This is my populate cache policy:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PopulateCache continueOnError="false" enabled="true" name="CP_StoreToken">
    <CacheKey>
        <KeyFragment ref="hash_token"/>
    </CacheKey>
    <Scope>Application</Scope>
    <ExpirySettings>
        <TimeOfDay ref="timeOfDay"/>
        <ExpiryDate ref="expiryDate"/>
    </ExpirySettings>
    <Source>as-response</Source>
</PopulateCache>

When i call Cache Lookup after the expiry date, it still found the token in cache.

I noticed that if i put a date in the past, and i try to lookup it, i can still found the entry key. Does Apigee not validate input in TimeOfDay and ExpiryDate parameters?

Can anyone help us? @dchiesa1 

Thanks

[dchiesa1 · Accepted solution]

Thanks for your question.

The documentation for PopulateCache does not make it clear, but you should use exactly one of TimeOfDay, TimeoutInSeconds, or ExpiryDate. This is a documentation bug! If you use more than one, there is an order of precedence: TimeoutInSeconds, ExpiryDate, TimeOfDay.

There is a second documentation bug: the documentation does not describe the behavior of the policy in the case in which you specify a date that occurs in the past. In this case, the policy coerces the TTL to be the maximum TTL, which is 30 days. I am not sure this confirms to the Principle of Least Astonishment, but regardless, that is the behavior, and we should document it.

I'll get the documentation fixed.

In my opinion the ExpirySettings should be a little more flexible, and allow you to express expiry in human-scale quantities, like "5d" to indicate 5 days, instead of 5*86400 = 432000. But it isn't that flexible now.

In summary, the policy seems to be working as intended. It is clear the documentation does not describe exactly what is intended in these edge cases.

I hope this gives you enough information to proceed with your implementation.

---

Update: while investigating this I have found a second, somewhat related bug, this time in the runtime. For each of the child elements, ExpiryDate, TimeoutInSeconds, and TimeOfDay, the configuration validation for X and hybrid allows either a text value or a ref attribute, but not both. It should allow both. The internal reference for this bug is: b/245398397.

<PopulateCache>
  ...
  <ExpirySettings>
    <!-- this is accepted as valid -->
    <ExpiryDate ref='date-here'/>

    <!-- this is accepted as valid -->
    <ExpiryDate>11-07-2022</ExpiryDate>

    <!-- currently, this is incorrectly rejected as invalid -->
    <ExpiryDate ref='date-variable'>11-07-2022</ExpiryDate>
  </ExpirySettings>
  ...

 For now, if you want to have the "fallback behavior" in which the policy uses the value in a variable, and if the variable is not defined, it uses a default value.... you will need to just use the ref variant, making sure to pre-Assign the default value to the referenced variable, in the null case. I hope this is clear. We'll get this fixed, but it's not a high priority since there's a workaround.

[Giupo]

Thank you very much, it's all clear to me now!