Errors during query execution

[kyotsu]

Hi, 

Recently, we've met several errors during query executions. There are two notable errors that our customers need some explanations about the causes of them: 

• An internal error occurred and the request could not be completed. This is usually caused by a transient issue. Retrying the job with back-off as described in the BigQuery SLA should solve the problem.
  
• Error encountered during execution. Retrying may solve the problem.

Can you please provide us with some information about these errors? Are those happened because of the large amount of data that the queries have to process?

Besides, we've noticed that in those error messages, the recommended solutions were implementing retry method with back-off. So is there any suggestion for the suitable wait-time between each retries ?

Best regards,
FDH team

[nceniza]

I believe this answer described the error better: https://www.googlecloudcommunity.com/gc/Data-Analytics/Continuously-receiving-quot-internal-error-qu...

It most likely that slots in bigquery are overloaded at the moment you ran the query and suggested to wait according to back-off requirements described in BQ SLA.

With back off requirements explained in this document:

"

• "Back-off Requirements" means, when an error occurs, the Customer Application is responsible for waiting for a period of time before issuing another request. This means that after the first error, there is a minimum back-off interval of 1 second and for each consecutive error, the back-off interval increases exponentially up to 32 seconds.

"

Reference: https://cloud.google.com/bigquery/sla#:~:text=%22Back-off%20Requirements