Dear Sir / Madam
I got a question about how BigQuery querying external table. I do very appreciate your kind help.
As per my understanding, when we are querying external table (permanent and temporary) from GCS data files in BigQuery, it's similar to Hadoop Hive to create schema only and querying data on the data file directly. BigQuery doesn't create an internal temp table and load data into the temp table and then just start to query.
Some of my colleagues told me that it can query data file directory in S3 bucket on AWS, but not GCP BigQuery. They said that when BigQuery querying external table from GCS, BigQuery create an internal table and load data into the temp table and then just start to query.
I am not sure which one is right and personally I disagree my colleagues' point due to that loading data from GCS to BigQuery takes time and network transferring. In addition, GCS and BigQuery are both with distribution storage. It's strange to transfer data between two distribution storage.
Thank you very much
Best regards,
LinkedIn
Twitter
