message "Got an error reading communication packets" in Aurora MySQL error.log

[masak1]

Hi there,

I'm building CDC flow with Datastream, from Aurora MySQL 3.05 to BigQuery.

After starting Datastream, I've found message "Got an error reading communication packets" in Aurora MySQL error.log as below:

2024-08-26T00:26:00.00000 xxx [Note] [MY-010914] [Server] Got an error reading communication packets

I believe that Datastream CDC access is causing this message.

The datastream itself is working fine without errors.

I'm trying change some system parameters of MySQL like connect_timeout, but, I cannot suppress this error.

Could you please tell me how to suppress or avoid the message? or Is this normal situation??

Thank you for your help in advance.

Masaki

[ms4446 · Accepted solution]

The error message in your Aurora MySQL error log indicates a temporary disruption in communication between the MySQL client, likely Datastream in this case, and the MySQL server. While such messages can occur intermittently due to network fluctuations or resource constraints, they should not be ignored if they persist.

To address this issue, start by investigating the stability of the network connection between Aurora MySQL and the Datastream service. Look for any signs of network congestion, latency, or firewall rules that could be interfering with communication. Additionally, monitor the CPU, memory, and disk utilization on your Aurora MySQL instance. High resource usage could contribute to these errors, and scaling up the instance or optimizing MySQL queries might be necessary to reduce the load.

It's also important to verify that the Datastream configuration is correctly set up and that it has the necessary permissions to access the MySQL binary logs. If you haven't already, consider cautiously tuning MySQL system variables like net_read_timeout, net_write_timeout, and max_allowed_packet to handle large data packets or slow network responses better.

If your setup includes the MySQL Connector/J library, ensure that you're using the latest version, as updates often include bug fixes and performance improvements that could reduce communication issues.

While this error in the log does not necessarily indicate a failure in the Datastream process, it's crucial to ensure that your CDC flow is functioning correctly without data loss or inconsistencies. If the problem persists despite these measures, contacting AWS Support with details of your configuration and the steps you've taken may provide further guidance tailored to your specific setup.

[masak1]

ms4446,

Thank you so much for your advice.

I have already adjusted the MySQL system variables, but the problem is still not resolved.

As you suggested, I will investigate the stability of the network connection.
I've set up two Steams with Aurora MySQL using "Forward SSH Tunnel" and "VPC Peering", and I've noticed some differences in how the errors occur.
It seems like there is some misconfiguration in the network between Datastream and Aurora MysQL.

Thank you for your help.

Masaki,