I am developing a JAVA application which will connect to BigQuery using JDBC and execute some SQL query to fetch data from tables. This project is in gradle. Can someone help me with documentation how can I import this JDBC dependency to my gradle JAVA project and create connection to BigQuery and execute query? I am very newbie in this field so I need some detailed answer.
Ensure the BigQuery API is enabled for your project by navigating to "APIs & Services" -> "Dashboard" -> "Enable APIs and Services". Search for and enable the "BigQuery API".
1. Obtain Service Account Credentials
2. Add JDBC Dependency in Gradle
build.gradle
file, add the Simba JDBC Driver for Google BigQuery:
dependencies {
implementation 'com.simba.googlebigquery.jdbc42.Driver:1.2.3' // Replace with the latest version
}
gradle dependencies
.3. Java Code to Connect and Query BigQuery
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
public class BigQueryConnection {
public static void main(String[] args) {
String projectId = "your-project-id";
String serviceAccountKeyPath = System.getenv("BIGQUERY_KEY_PATH");
String query = "SELECT * FROM `your-project-id.your-dataset.your_table` LIMIT 10";
try (Connection conn = DriverManager.getConnection(
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=" + projectId +
";OAuthType=0;OAuthServiceAcctEmail=your-service-account-email;OAuthPvtKeyPath=" + serviceAccountKeyPath)) {
ResultSet resultSet = conn.createStatement().executeQuery(query);
while (resultSet.next()) {
System.out.println(resultSet.getString(1)); // Example processing
}
} catch (SQLException e) {
System.err.println("Error connecting to BigQuery: " + e.getMessage());
}
}
}
Important Notes
Have you tried this code block. Because I can see that my gradle project is not able to find and download this dependency.
Error : Could not find com.simba.googlebigquery.jdbc42.Driver:1.5.2:
version 1.5.2 has been released and this is release note for this. https://storage.googleapis.com/simba-bq-release/jdbc/release-notes_1.5.2.1005.txt
I can see there is one open issue that maven and gradle dependency for this is also not available.
https://issuetracker.google.com/issues/180413368?pli=1
Here is how to troubleshoot the issue:
1. Verify Version and Repository
2. Alternative Approach: Manual Download and Addition
If the driver isn't found automatically, try this workaround:
libs
directory at the root of your project.libs
directory.repositories {
flatDir {
dirs 'libs'
}
}
dependencies {
implementation(name: 'simba-googlebigquery-jdbc42', version: '1.5.2', ext: 'jar')
}
Note: Match the name
attribute in the dependency to the JAR filename (without version or extension), unless you specifically rename the file.
3. Investigate the Google Issue Tracker