bigquery - on-demand vs enterprise edition auto-scaler

[crazyrevol]

We're using on-demand bigQuery model exclusively for all our projects. my observation is that in on-demand model, as long as we're below 2000 slots per project, we don't see any wait times in stages of jobs running in our project. So, I would consider the slots availability in on-demand model as HOT even though Google says it depends on availability 

If we switch to enterprise edition, and go for 0 to 2000 slots autoscaling, should I expect any wait-times increase? Google says your baseline slots are HOT and readily available.

[MarvinLlamas · Accepted solution]

Hi @crazyrevol,

Welcome to the Google Cloud Community!

It appears you're considering the trade-offs of moving from BigQuery's on-demand pricing to Enterprise Edition autoscaling (0–2000 slots). While dedicated slots offer advantages, you’re considering whether this shift could lead to longer wait times and impact query performance. Your objective is to find the optimal balance between cost efficiency and reliable performance.

Here are the potential ways that might help with your use case:

• Consider a minimum baseline: Rather than starting with a range of 0-2000 slots, establish your own small baseline allocation (e.g., 100-200 slots). This ensures your system has a consistently available pool, minimizing initial wait times while still leveraging autoscaling for handling larger demand surges.

• Monitor Performance: After switching, track your job performance by analyzing queue and execution times. This helps assess the autoscaler's impact, which you can compare against historical on-demand data.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[crazyrevol]

Hello @MarvinLlamas 

Thank you for your reply! 

The monitor Performance link you provided above, is not for BigQuery as fas as I understand. Is there is UI monitoring way I can set alerts if any bigquery job is taking longer than say twice or 10 times longer than historical average ?