Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

BigQuery Syntax Change -- SELECT DISTINCT json_col no longer works?

I have an hourly dbt job running. Last night, between 3am-4am, the job started failing and 9 files/models started producing errors.

My dbt workflow ingests JSON data from MongoDB and saves the output to Google BigQuery tables. Last night, suddenly, the code in these 9 files started producing the error: Column of type JSON cannot be used in SELECT DISTINCT.

My code in these 9 files takes on the form:

SELECT DISTINCT col1, col2, json_col3
…

There are many JSON type columns in my data and I only unnest some of them. This worked previously without issue in 9 files, for months. Now, there is a problem. Was there a change to Google BigQuery that no longer allows SELECT DISTINCT with JSON columns? I don’t see any mention of such a change in the release notes, nor any release notes for today.

 
0 1 3,509
1 REPLY 1