Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Problem with partitioned table in BigQuery and streaming buffer from Datastream

Hello,

I have created a datastream from MySql to BigQuery, a made table in Bigquery partitioned.

Screenshot 2024-02-17 at 23.11.21.png

It looks like inserting data works fine with partitions.

But when I try to make a query, while there are still some data in the streaming buffer, it doesn't use partitions and read data from the whole table.

Screenshot 2024-02-17 at 23.12.32.png

When there are no more data in streaming buffer, it works fine.

Screenshot 2024-02-17 at 23.23.14.png

Why it has to read whole table? Shouldn't it read only the required partitions + streaming buffer?

1 10 3,476
10 REPLIES 10