This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Data Quality Scan in Dataplex with Python library

Posted on 10-24-2023 06:57 AM
Wolfmarel
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Hello!

I am configuring the creation of Data Quality scan using the python library to interact with the Dataplex API (dataplex_v1). I need, when creating a scan, to configure the option to have the results published in BigQuery and Dataplex Catalog UI. Does anyone know how to do it?

2 3 1,141
Topic Labels
3 REPLIES 3

Posted on --/--/---- --:-- AM
SebastiandMV
Bronze 3
Bronze 3
Reply posted on --/--/---- --:-- AM

I face the same problem. As far as I know, it is not yet possible to publish DataScan DataQuality results to the BigQuery UI through Python code. However, I'm not 100% percent sure, but there may be a way to do this through Apache Airflow or other APIs or libraries.  I'm currently investigating how to do it. If I find out anything, I will let you know!

Posted on --/--/---- --:-- AM
Wolfmarel
Bronze 2
Bronze 2
In response to SebastiandMV
Reply posted on --/--/---- --:-- AM

Thank you very much Sebastian, I'll keep on investigating and I'll comment if I get the solution.

Posted on --/--/---- --:-- AM
BhrugeshJadav
Bronze 1
Bronze 1
In response to SebastiandMV
Reply posted on --/--/---- --:-- AM

Hi Sebastian,

Did you find any solution to create a data quality scan using Python? I'm trying to achieve the same thing but haven't been able to find a solution yet.

0 Likes
Top Solution Authors
View all