Publish Data Quality Scan to Dataplex Catalog UI with Python

[dacong]

When I create a data scan in python, I want to publish it to the Dataplex Catalog UI interface at the same time. What should I do?

[caryna]

Hi @dacong,

Welcome to Google Cloud Community!

Currently, there is no option for you to directly execute Python code within the Dataplex Catalog UI and there is no available documentation regarding this. The Dataplex Catalog UI primarily serves as a visual interface and does not support direct code execution.

However, you can check out a YouTube video that explains Automated Data Profiling and Quality Scans via Dataplex. It demonstrates how to use the gcloud CLI to import YAML files and create data quality scans in Google Cloud Dataplex.

Note: Make sure to enable Dataplex API before proceeding.

I hope the above information is helpful.

[dacong]

Thank you for your answer. I want to implement the operation of publishing to UI in Python code, as shown in the figure

[kolban]

My interpretation of your question is as follows:

Dataplex Catalog is metadata management system built into Google Cloud Platform.  It tracks the existence of data assets as well as storing/editing metadata associated with those assets.  The data that is stored in Dataplex Catalog can be placed there by running tools (eg. Dataplex Quality Scans) or by creating assets (eg. create a new table in BigQuery).  In addition, the data associated with Dataplex Catalog can be entered/edited through the Console UI.

When you say "I want to publish to the UI from Python code", I am interpreting that as "I want to add new or update existing Dataplex Catalog maintained data using code that I write in the Python programming language".   So we aren't really "publishing" anything nor are we "writing to the UI" ... instead we are using Python to edit/create Dataplex Catalog managed data.

We can do this using the Python Client for Cloud Dataplex (ref).  Using this API, we can start working with Dataplex Catalog entries ... (ref).

[chaiT899]

Hey, I am also trying to create data profile scans using the dataplex library and want  to enable publishing to bigquery and catalog  ui.  Did you find a solution?