Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Getting a lot of MYSQL_DECODE_ERROR with Datastream on a MySQL to BigQuery stream

hello,

since 1 or 2 weeks i'm getting more and more MYSQL_DECODE_ERROR on 2 of my 3 Datastreams.

Each being MySQL to BigQuery with the same data schema.

1 example:

 

{
  "insertId": "xxxxxxxxx-xxx-xxxx-xxxx-xxxxxxxxx",
  "jsonPayload": {
    "context": "CDC",
    "object_name": "xxxxxxxxx",
    "message": "Discarded 2 unsupported events with reason code: MYSQL_DECODE_ERROR. Latest discarded event details: Discarded an event from xxxxxxxxx.xxxxxxxxx: Unable to decode column xxxxxxxxx.xxxxxxxxx with character_set_name utf8mb3 for value b'processing'.",
    "read_method": "mysql-cdc-binlog",
    "event_code": "UNSUPPORTED_EVENTS_DISCARDED"
  },
  "resource": {
    "type": "datastream.googleapis.com/Stream",
    "labels": {
      "location": "europe-west1",
      "resource_container": "projects/xxxxxxxxx",
      "stream_id": "xxxxxxxxx"
    }
  },
  "timestamp": "2023-03-20T05:49:09.138496Z",
  "severity": "WARNING",
  "logName": "projects/xxxxxxxxx/logs/datastream.googleapis.com%2Fstream_activity",
  "receiveTimestamp": "2023-03-20T05:49:09.506083070Z"
}

 

It's beginning to represent most of the transactions, and only initiating backfills seems to temporarilly catch up to the source data. 

The charset in my MySQL table definitions being plain utf8, and that issue appearing out of the blue on working Datastreams (before) I have no idea where to look.

Has anyone had the case ?

0 1 392
1 REPLY 1

I just found this bug on the issue tracker which perfectly describes mine : https://issuetracker.google.com/issues/247028143

Which reminds me, i forgot to mention that i went through an upgrade of MySQL version from 5.x to 8.x at the same period...

Top Solution Authors