Google cloud fusion with MongoDB

[mani_ranjan]

Hi,

 I am trying to connect  google cloud fusion to mongoDB as source . MongoDB Atlas is hosted on Google cloud . While running the pipeline I am getting java.net.UnknownHostException. I am able to connect to Mongo using compass tool and many other java/Node applications are able to connect to the DB . Any idea if we need to make any specific format in connection string . TIA

[ms4446]

The java.net.UnknownHostException error in Data Fusion when connecting to MongoDB Atlas suggests a problem with DNS resolution within the Data Fusion environment. While your other applications can connect to MongoDB Atlas using the SRV connection string (mongodb+srv://), Data Fusion might not support the necessary DNS SRV record lookups required by this format.

MongoDB Atlas's SRV connection strings rely on DNS to resolve the cluster's hostnames and discover configuration details. If the environment running the Data Fusion pipeline lacks the capability to perform these DNS lookups, it won't be able to resolve the host, leading to the UnknownHostException.

To resolve this issue, you should switch to the standard connection string format that starts with mongodb://. This format explicitly specifies the hostnames and ports of your MongoDB cluster nodes, eliminating the need for DNS SRV resolution. You can obtain this standard connection string from the MongoDB Atlas dashboard:

1. Navigate to your cluster in MongoDB Atlas.
2. Click on "Connect" and choose "Connect Your Application".
3. Select your driver and version.
4. Copy the standard connection string provided (it begins with mongodb://).

Replace the SRV connection string in your Data Fusion pipeline with this standard one. This change should allow Data Fusion to resolve the hostnames correctly and establish a connection to MongoDB Atlas.

Additionally, ensure that the network environment of your Data Fusion instance has the necessary permissions and configurations to access MongoDB Atlas. Check for any firewall rules or VPC configurations that might restrict outbound traffic to the MongoDB Atlas servers. If you're using VPC peering or private endpoints, verify that these are set up correctly.