BigtableTableAdminClient getTableStats not found in artifact google-cloud-bigtable jar

[viveksharma0wmt]

I am looking for the same thing asked in this thread : Is there a Java API to fetch Bigtable table stats .

However, I could not find that method in BigtableTableAdminClient bundled with latest version of this jar artifact at this point :

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-bigtable</artifactId>
  <version>2.39.4</version>
</dependency>

Can someone point me to the right artifact to include in my pom ?

I want to run this same code against my Bigtable table:

    BigtableTableAdminClient client = BigtableTableAdminClient.create();

    // Get the table stats for the table named "my-table".
    TableStats tableStats = client.getTableStats(GetTableStatsRequest.newBuilder()
        .setName("projects/my-project/instances/my-instance/tables/my-table")
        .build());

 Regards,

Vivek.

[ms4446]

The getTableStats method you're looking for is not directly available in the google-cloud-bigtable library as of version 2.39.4. However, you can still get information about your Bigtable tables using the following approaches:

1. Basic Table Information:

You can use the BigtableTableAdminClient class to retrieve basic information about your table, such as its ID and column families. Here's how:

import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient;
import com.google.cloud.bigtable.admin.v2.models.Table;
import com.google.cloud.bigtable.admin.v2.models.TableName;

public class BigtableTableInfo {
    public static void main(String[] args) {
        // Replace with your actual project ID, instance ID, and table ID
        String projectId = "your-project-id";
        String instanceId = "your-instance-id";
        String tableId = "your-table-id";

        try (BigtableTableAdminClient adminClient = BigtableTableAdminClient.create(projectId, instanceId)) {
            TableName tableName = TableName.of(projectId, instanceId, tableId);
            Table table = adminClient.getTable(tableName);

            System.out.println("Table ID: " + table.getId());
            System.out.println("Column Families: " + table.getColumnFamilies());
            // You can print other table details as needed
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

2. Detailed Table Statistics:

To get more comprehensive statistics about your Bigtable table, such as row count, size, and throughput, you'll need to use Google Cloud Monitoring. The google-cloud-monitoring library provides APIs to fetch these metrics. Here's an example:

import com.google.cloud.monitoring.v3.MetricServiceClient;
import com.google.monitoring.v3.ListTimeSeriesRequest;
import com.google.monitoring.v3.ProjectName;
import com.google.monitoring.v3.TimeInterval;
import com.google.monitoring.v3.TimeSeries;
import com.google.protobuf.util.Timestamps;

import java.io.IOException;

public class BigtableMetricsExample {
    public static void main(String[] args) {
        // Replace with your actual project ID
        String projectId = "your-project-id";

        try (MetricServiceClient metricServiceClient = MetricServiceClient.create()) {
            ProjectName projectName = ProjectName.of(projectId);
            // Set the time interval for the metrics you want to fetch
            long startMillis = System.currentTimeMillis() - 3600000; // 1 hour ago
            long endMillis = System.currentTimeMillis();

            TimeInterval interval = TimeInterval.newBuilder()
                    .setStartTime(Timestamps.fromMillis(startMillis))
                    .setEndTime(Timestamps.fromMillis(endMillis))
                    .build();

            ListTimeSeriesRequest request = ListTimeSeriesRequest.newBuilder()
                    .setName(projectName.toString())
                    .setFilter("metric.type=\"bigtable.googleapis.com/instance/table/stats\"")
                    .setInterval(interval)
                    .build();

            MetricServiceClient.ListTimeSeriesPagedResponse response = metricServiceClient.listTimeSeries(request);
            for (TimeSeries ts : response.iterateAll()) {
                System.out.println(ts); 
                // Process the TimeSeries object to extract the metrics you need
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

• Dependencies: Make sure to include both the google-cloud-bigtable and google-cloud-monitoring dependencies in your project's pom.xml file.
• Authentication: Ensure that your application has the necessary permissions to access Bigtable and Google Cloud Monitoring resources.

Dependencies in pom.xml:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-bigtable</artifactId>
  <version>2.39.4</version>
</dependency>
<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-monitoring</artifactId>
  <version>2.1.8</version>
</dependency>

[viveksharma0wmt]

I tried with above example for #2 (detailed stats) that you mentioned, but when I run it I am seeing an error:

com.google.api.gax.rpc.NotFoundException: io.grpc.StatusRuntimeException: 
NOT_FOUND: Cannot find metric(s) that match type = "bigtable.googleapis.com/my-instance/my-table/stats".

 I checked that I do have the required permissions (roles/monitoring.admin)

Am I missing some other setting to be done on the table to enable these monitoring stats ?

[zijieli · Accepted solution]

The gcloud command is the only option to view the table stats now. You can open a FR or PR for Bigtable Java client support for table stats.

[viveksharma0wmt]

Tried this but I am getting this error:
ERROR: (gcloud.bigtable.instances.tables.describe) PERMISSION_DENIED: Access denied. Missing IAM permission: bigtable.tables.get.

Although I checked that the command I used to set the service account (that account has the permission : roles/bigtable.admin😞

gcloud auth activate-service-account --key-file=my-json

I am still getting the above error. 

[viveksharma0wmt]

Okay what worked is this command for me:

gcloud auth activate-service-account --key-file=mine

and then

gcloud bigtable instances tables describe my-table --project=my-project --instance=my-instance --view stats