This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

BigQuery data random after data loading

Posted on 11-07-2024 07:34 PM
Xedonedron
Bronze 1
Bronze 1
Reply posted on --/--/---- --:-- AM

I'm building an ETL from  by using Cloud Composer/Airflow and GoogleCloudStorageToBigQueryOperator. Wether the file is CSV or Parquet, loading it using WRITE_TRUNCATE method, the row is randomize. For example:

100 row from the raw CSV with sequential row id such as 1, 2, 4, 5, 6, then load to the BigQuery, and then get random like it can be start from 50, 22, 15, 59, 5, 1, 7 etc.

Any solve for this issue or how to avoid this?

0 3 173
Topic Labels
0 Likes
3 REPLIES 3

Posted on --/--/---- --:-- AM
mars124
Silver 1
Silver 1
Reply posted on --/--/---- --:-- AM

Why are you trying to avoid this?

0 Likes

Posted on --/--/---- --:-- AM
Xedonedron
Bronze 1
Bronze 1
In response to mars124
Reply posted on --/--/---- --:-- AM

Sure I need avoid this because the data is not order properly.

0 Likes

Posted on --/--/---- --:-- AM
mars124
Silver 1
Silver 1
In response to Xedonedron
Reply posted on --/--/---- --:-- AM

Why do you need to have them ordered "properly" in the table?

0 Likes