Hello, I am new to Data Fusion, and I am having an issue getting a pipeline up and running. I am trying to move data from a CloudSQL MySQL instance to Big Query. In the pipeline I have tried using the MySQL, CloudSQL MySQL and Database sources, but I get the same error each time:
Database Source:
Spark program 'phase-1' failed with error: Plugin with id Database:source.jdbc.mysql does not exist in program phase-1 of ap
plication gs_test_two.. Please check the system logs for more details.
MySQL Source:
Spark program 'phase-1' failed with error: Plugin with id MySQL2:source.jdbc.mysql does not exist in program phase-1 of application gs_test_two.. Please check the system logs for more details.
CloudSQLMySQL Source:
Spark program 'phase-1' failed with error: Plugin with id CloudSQL MySQL:source.jdbc.cloudsql-mysql does not exist in program phase-1 of application gs_test_two.. Please check the system logs for more details.
So as you can see, basically the same error each time.
I know the connections work, as I can look up the MySQL Databases and see table schemas and data through them. What could I be doing wrong here? It's like the pipeline isn't talking to the connections properly.
The instance is on a dedicated VPC with private IP, we have a VM running cloudSQL proxy, private IP is enabled on the database and is peered to the same VPC.
Thanks.
LinkedIn
Twitter
