Filtering Static Files Before Callout with traffic extension

[laurodd]

Hello everyone, I’d like to get your input on something I’ve been testing.

I was exploring the possibility of extending the match condition in our service extension (traffic) to filter out static files using CEL (Common Expression Language). The idea is to avoid calling our VM/Cloud Run for these requests, since our bot protection solution currently ignores static files (FYI, this logic is currently handled there).

During testing I noticed:

• CEL expressions seem to be limited to 512 characters (returns INVALID_CEL_EXPRESSION when exceeded).

  • Regex is not supported with CEL, if I am not mistaken

Given this, is there any way we can extensively ignore static file requests before they reach the callout (VM/Cloud Run), so we can bypass the full logic for these cases?

Below is the CEL expression I used for testing (mainly hardcoded), It covers only a partial list of static file types for now.

Thanks

!(
  request.path.endsWith(".avi")   ||
  request.path.endsWith(".avif")  ||
  request.path.endsWith(".bmp")   ||
  request.path.endsWith(".css")   ||
  request.path.endsWith(".eot")   ||
  request.path.endsWith(".flac")  ||
  request.path.endsWith(".flv")   ||
  request.path.endsWith(".gif")   ||
  request.path.endsWith(".gz")    ||
  request.path.endsWith(".ico")   ||
  request.path.endsWith(".jpeg")  ||
  request.path.endsWith(".jpg")   ||
  request.path.endsWith(".js")    ||
  request.path.endsWith(".json")  ||
  request.path.endsWith(".less")  ||
  request.path.endsWith(".map")   ||
  request.path.endsWith(".mka")   ||
  request.path.endsWith(".mkv")   ||
  request.path.endsWith(".mov")   ||
  request.path.endsWith(".mp3")   ||
  request.path.endsWith(".mp4")   ||
  request.path.endsWith(".mpeg")  ||
  request.path.endsWith(".mpg")   ||
  request.path.endsWith(".ogg")   ||
  request.path.endsWith(".ogm")   ||
  request.path.endsWith(".opus")  ||
  request.path.endsWith(".otf")   ||
  request.path.endsWith(".png")   ||
  request.path.endsWith(".svg")   ||
  request.path.endsWith(".svgz")  ||
  request.path.endsWith(".swf")   ||
  request.path.endsWith(".ttf")   ||
  request.path.endsWith(".txt")   ||
  request.path.endsWith(".wav")   ||
  request.path.endsWith(".webm")  ||
  request.path.endsWith(".webp")  ||
  request.path.endsWith(".woff")  ||
  request.path.endsWith(".woff2") ||
  request.path.endsWith(".xml")   ||
  request.path.endsWith(".zip")
)

[kensan · Accepted solution]

Hi @laurodd ,

Welcome to Google Cloud Community.

CEL uses a subset compatible with RE2 so some advanced REGEX features are not supported. This page lists the regular expression syntax accepted by RE2. In that case you can use the matches() function so you can shorten your expression and will not hit the 512 character limit. 

Here is the sample CEL expression would look like:

!request.path.matches('.*\\.(avi|avif|bmp|css|eot)$’)

This can be easily maintained by editing the list inside the parentheses to add or remove a file type.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[laurodd]

Thanks a lot for your help @kensan . I really appreciate it!

Totally missed that possibility on the docs, this is exactly what are doing on our go module and it will fit perfectly on the traffic extension configuration.

Best regards, Lauro

[laurodd]

Just to give you an update @kensan: it worked as expected with a small regex, but with multiple extensions (just like we use in our callout) we have an error.

I understand that this is probably related to performance and how to treat the requests in real-time. If you have other suggestion, please let me know!

Service Extension update failed: The request was invalid: INVALID_CEL_EXPRESSION: INVALID_CEL_EXPRESSION: regex length cannot be more than 100)

!request.path.matches('.*\\.(avi|avif|bmp|css|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|json|less|map|mka|mkv|mov|mp3|mp4|mpeg|mpg|ogg|ogm|opus|otf|png|svg|svgz|swf|ttf|wav|webm|webp|woff|woff2|xml|zip)$')