I have an hourly dbt job running. Last night, between 3am-4am, the job started failing and 9 files/models started producing errors.
My dbt workflow ingests JSON data from MongoDB and saves the output to Google BigQuery tables. Last night, suddenly, the code in these 9 files started producing the error: Column of type JSON cannot be used in SELECT DISTINCT.
My code in these 9 files takes on the form:
SELECT DISTINCT col1, col2, json_col3 …
There are many JSON type columns in my data and I only unnest some of them. This worked previously without issue in 9 files, for months. Now, there is a problem. Was there a change to Google BigQuery that no longer allows SELECT DISTINCT with JSON columns? I don’t see any mention of such a change in the release notes, nor any release notes for today.