Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Help me understand sudden drop in BI Engine used bytes

I have a large table  (~150TB) and a 100 GB reservation for BI Engine. The table was constantly using the BI Engine but suddenly stop using it. 

Screenshot 2023-10-31 at 8.08.49 AM.png


 I've observed this behaviour a couple of time in the last few days. Whenever this happens, all the queries on the table scan complete table instead of reading data from the BI Engine, which increase bytes_scanned as well as slotms.

My current understanding is BI Engine keeps table data in memory for 24 hours if it's not queried. I am not seeing any changes in the query patterns so I don't understand why there would be such a large drop in BI Engine usage.  I've also tried to add tables as `preferred tables` for the BI reservation.

Any help on how BI Engine decides to remove table blocks from memory would be useful.

0 3 290
3 REPLIES 3