This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Data Fusion pipeline Failing with internal server error when tried to read data from Bigquery

Posted on 07-21-2023 11:41 AM
mani_techo
Bronze 1
Bronze 1
Reply posted on --/--/---- --:-- AM

I have created a CDC streaming pipeline using datastream to read data from Oracle to bigquery by setting staleness limit to 0 in datastream I am getting the following error (1) when I tried to read data from sink table in big query 

I am not facing this issue in tables which are configured staleness greater than zero in data stream

If it is a bug solve it or raise proper exception in bigquery during this scenario

1.

mani_techo_0-1689964800155.png

 

0 1 273
Topic Labels
0 Likes
1 REPLY 1

Posted on --/--/---- --:-- AM
ms4446
Staff
Reply posted on --/--/---- --:-- AM

The error message you're encountering suggests an internal issue with BigQuery. This could be due to a variety of factors, including an issue with your specific BigQuery dataset or table.

Here are some steps to help you troubleshoot this error:

  1. Review your BigQuery dataset and table permissions. Ensure that you possess the required permissions to access data from your dataset and table.
  2. Consider retrying the job with a back-off strategy. This involves reattempting the job with a delay between each try, allowing the BigQuery service some time to recover from the error.

Additionally, you might want to:

  • Attempt to create a new CDC streaming pipeline from the beginning.
  • If you're using a custom connector, make sure it's updated.