Getting SIGSEGV error when trying to connect to bigquery from spark jar

[ashutosh2]

I am getting SIGSEGV error when trying to create a connection from within spark jar to bigquery.  I am running the spark jar on dataproc cluster. I am getting error when trying: datasource.getConnection().

This is the full error:

#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f4eca441c80, pid=28494, tid=0x00007f4f397b3700
#
# JRE version: OpenJDK Runtime Environment (8.0_322-b06) (build 1.8.0_322-b06)
# Java VM: OpenJDK 64-Bit Server VM (25.322-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C 0x00007f4eca441c80
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /tmp/58aa059c-a453-4ce9-9401-61ae287d0e6d/hs_err_pid28494.log
#
# If you would like to submit a bug report, please visit:
# https://github.com/adoptium/adoptium-support/issues
#

[oakinlaja]

Hello,

I would like if you can provide more information about how you are trying to create the Connection. As you may know there might be a number of reasons for SIGSEGV error. For example, on this report[0], the issue was a result of an internal issue. However, before we go there, Can you provide more information on your implementation? Are you using the BigQuery Connection API[this is better suited for external Data Sources][1] or you are using the BigQuery connector with Spark[which I think would be better suited for your use-case, Spark running in Dataproc][2].

If you are following any documentation, please let us know.

[0]https://github.com/GoogleCloudDataproc/spark-bigquery-connector/issues/112
[1]https://cloud.google.com/bigquery/docs/working-with-connections#java
[2]https://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example#pyspark

[ashutosh2]

Hi, thanks for your reply. The issue is resolved now, it was due to netty-shared jar in dependency: 

https://github.com/netty/netty/issues/11879.

I needed to delete some data from bigquery table, thats why I couldn't use spark bigquery connector. I had to create bigquery jdbc connection, this is the sample code I was following:

public class ConnectServiceAuthentication {
public static Connection connectViaDS() throws SQLException {
    Connection connection = null;
    DataSource ds = new 
com.simba.googlebigquery.jdbc.DataSource();
    ds.setURL("jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;\n" + "OAuthType=3;ProjectId=myProjectid;");
    ds.setProjectId("myProjectid");
    ds.setOAuthType(0); // Service Authentication
ds.setOAuthServiceAcctEmail("myproject_bigquery@myproject.iam.gserviceaccount.com");
    ds.setOAuthPvtKey("/home/key.json");
    connection = ds.getConnection();
    return connection;
}

public static void main(String[] args) throws SQLException {
    Connection connection = connectViaDS();
    String sql = "select count(1) from  temp.tempn1;";

    PreparedStatement ps = connection.prepareStatement(sql);
    ResultSet rs = ps.executeQuery();
    if (rs.next())
        System.out.println(rs.getInt(1));
    
    connection.close();
}