Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Data Fusion replication error

I updated my version of my data fusion instance and the process I had from replicating a sql server database to big query now does not work and gives me this error (imagen below), does anyone know what I should modify? I already tried creating another replication task from scratch but it doesn't work either. 

Screenshot 2024-08-16 223800.png

thanks a lot for helping me.

0 1 109
1 REPLY 1

Hi @JaimeEduardo,

Welcome to the Google Cloud Community!

It appears you have configured internal.key.converter and internal.value.converter, which were removed in version 2.0.0. This has led to compatibility issues with the previous replication process. It was recommended switching to the JSON converter and setting schemas.enable to false, as outlined in the Kafka documentation. It is advised to review the Kafka documentation and remove the deprecated properties.

Screenshot 2024-08-21 9.29.55 PM.png

However, the second warning message suggests that the plugin.path property might contain a variable, which could lead to unexpected behavior, or it may be empty or misconfigured. As mentioned in Kafka documentation, avoid using config provider variables in plugin.path because the worker's scanner uses the raw path before config providers are initialized. It is also advisable to use fixed or absolute paths, or ensure that any variables are fully initialized before being used in the plugin.path property.

Example: plugin.path=/usr/local/share/java,/usr/local/share/kafka/plugins,/opt/connectors

Screenshot 2024-08-21 9.30.15 PM.png