Error with Extension for "CSV Custom IOC" Parser

[migueltubia]

Hi!

I'm doing an extension for "CSV Custom IOC" Parser, to parser correctly a IoC Feed. The statedump shows the data but I get this result:

"No UDM events or entities were generated for the current parser configuration. If this is not intended, rectify the code snippet/UDM mappings and then click preview."

The statedump shows this output:

  "@output": [
    {
      "idm": {
        "entity": {
          "entity": {
            "ip": "129.232.208.18"
          },
          "metadata": {
            "entity_type": "IP_ADDRESS",
            "product_name": "Custom IOC",
            "threat": [
              {
                "severity": "LOW"
              }
            ],
            "vendor_name": "Custom IOC"
          }
        },
        "read_only_udm": {
          "metadata": {
            "event_timestamp": 1732005933,
            "event_type": "GENERIC_EVENT",
            "product_name": "Custom IOC",
            "vendor_name": "Custom IOC"
          }
        }
      },
      "ioc": {
        "categorization": "IP_ADDRESS",
        "confidence_score": "100",
        "feed_name": "Custom IOC (CSV)",
        "ip_and_ports": {
          "ip_address": "129.232.208.18"
        }
      },
      "udm": {
        "principal": {
          "ip": "129.232.208.18"
        }
      }
    }
  ],

And the extension code is the following:

filter {
    csv {
        source => "message"
        #columns => ["category", "value", "score", "severity_number"]
    }

  if [column3] == "" {
    mutate {
        replace => { "column3" => "0" }
    }
}

  mutate {
    convert => { 
      "column4" => "integer"
     }
  }

  mutate {
    replace => {
      "event1.idm.entity.metadata.product_name" => "Custom IOC"
      "event1.idm.entity.metadata.vendor_name" => "Custom IOC"
   "event1.idm.read_only_udm.metadata.product_name" => "Custom IOC"
   "event1.idm.read_only_udm.metadata.vendor_name" => "Custom IOC"
      "event1.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT"
     }
  }

   mutate {
    copy => {
      "event1.idm.read_only_udm.metadata.event_timestamp" => "@createTimestamp.seconds"
    }
  }

  mutate {
      replace => {
        "event1.ioc.feed_name" => "Custom IOC (CSV)"
      }
    }


    mutate {
      replace => {
        "event1.ioc.confidence_score" => "%{column3}"
        "event1.ioc.categorization" => "%{column1}"
      }
    }

  if ([column1] =="IPv4-Addr") {

    mutate {
      replace => {
        "event1.idm.entity.metadata.entity_type" => "IP_ADDRESS"
        "event1.ioc.categorization" => "IP_ADDRESS"
        "event1.idm.entity.entity.ip" => "%{column2}"
        "event1.udm.principal.ip" => "%{column2}"
        "event1.ioc.ip_and_ports.ip_address" => "%{column2}"
      }
    }
  }

  if [column4] >= 90 {
    mutate {
        replace => {
            "threat_det.severity" => "CRITICAL"
        }
    }
  } else if [column4] >= 70 {
    mutate {
        replace => {
            "threat_det.severity" => "HIGH"
        }
    }
  } else if [column4] >= 40 {
    mutate {
        replace => {
            "threat_det.severity" => "MEDIUM"
        }
    }
  } else if [column4] > 0 {
    mutate {
        replace => {
            "threat_det.severity" => "LOW"
        }
    }
  } else {
    mutate {
        replace => {
            "threat_det.severity" => "INFORMATIONAL"
        }
    }
  }

  mutate {
     merge => {
        "event1.idm.entity.metadata.threat" => "threat_det"
     }
  }

  mutate {
    merge => {
      "@output" => "event1"
    }
  }
  
  statedump{}
}

Any idea?

Thanks!!

M.

[Ben_T]

A few additions that might help troubleshoot development are initializing "threat_det" before your conditional statements as well as adding "on_error" blocks for better error handling.

You can find other troubleshooting tips here:

https://cloud.google.com/chronicle/docs/event-processing/parser-tips-troubleshooting

Hope this helps.

[migueltubia]

Hi @ben !

Thanks for your help. I have initializated the variable "threat_det" and added "on_error" statements, with no luck.

I have read the link in your reply, but I don't know how to debug or which test I can do...

Any idea? 🙂

Thanks!!