I'm trying to connect to Snowflake from Google Cloud Dataproc Serverless (Batch) Spark job (Spark 3.1 on Scala 2.12, Snowflake JDBC 3.13.30, Spark Snowflake connector 2.11.3) but I get connectivity issues:
23/05/03 18:49:45 ERROR RestRequest: Stop retrying since elapsed time due to network issues has reached timeout. Elapsed: 120,138(ms), timeout: 60,000(ms)
Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Connect to XXX.snowflakecomputing.com:443 [XXX.snowflakecomputing.com/34.107.221.154] failed: connect timed out.
I tried running it using default Dataproc solution (the one where you create clusters yourself) and there the problem doesn't occur there. Are there additional firewall rules which prevent Spark Serverless from reaching the outside world?
LinkedIn
Twitter
