BigQuery API question

[EyalMaoz]

Hi,

I am trying to get information about my datasets and tables using the BigQuery API - https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list

For my understanding the only way to get information about all my tables is to ListDatasets ->  than for each dataset need to ListTables -> than for each table id need to GetTable.

The property I am trying to get is "encryptionConfiguration" (of table, not dataset).

Because of a huge amount of tables I get throttling errors (which make sense).

So my question - is there a different way to get this information? maybe a a call to fetch all the tables of a dataset at once? or any other idea how to avoid this throttling issue.

Thank you..

[sarder]

Hello, 

I understand you are trying to get metadata of your Datasets and particularly, of tables via the bigQuery API. 

I think you can generally get metadata of your BigQuery tables[and others, like Datasets] from the Information_schema[0][1].  However, while it seems possible to get information about BigQuery Datasets[2][3] via the BigQuery API, this article[0] seems to suggest that metadata of the BigQuery tables can only be pulled via the Console or using the bq Query commands. Using this method, tables.get[4] only returns the table resource, which describes the structure of the table.

That said, specifically about getting the Custom encryption configuration, I believe you can pull information such as kmsKeyName[5], that is, the name of the Cloud KMS key used to encrypt the table, from the Information_schema for the Tables[0]. 

[0]https://cloud.google.com/bigquery/docs/information-schema-tables

[1]https://cloud.google.com/bigquery/docs/information-schema-intro

[2]https://cloud.google.com/bigquery/docs/dataset-metadata#api

[3]https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets/get

[4]https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get

[5]https://cloud.google.com/bigquery/docs/information-schema-tables#options_table