This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

DataplexServiceClient fails with google.api_core.exceptions.MethodNotImplemented

Posted 4 weeks ago
danielc_68
Bronze 1
Bronze 1
Reply posted on --/--/---- --:-- AM

Hi!

I'm struggling with using DataplexServiceClient. It fails with google.api_core.exceptions.MethodNotImplemented: 501 Received http2 header with status: 404 on any call I tried (get_asset, list_lakes, create_asset). I can use other APIs (e.g. listing buckets in storage.Client() works like a charm, but also dataplex_v1.CatalogServiceClient() seems to be working fine for me). I suspect some stupid misconfiguration on my side, but couldn't google any similar problem. 

Example code snippet:

dataplex_client = dataplex_v1.DataplexServiceClient()
res = dataplex_client.list_lakes(list_lakes)
res.result()
print(f"Result: {res}")

Exception is raised on res = dataplex_client.list_lakes(list_lakes) line.

Any thoughts what might be wrong?

0 2 140
0 Likes
2 REPLIES 2

Posted on --/--/---- --:-- AM
ruthseki
Staff
Reply posted on --/--/---- --:-- AM

Hi @danielc_68,

Welcome to Google Cloud Community!

A 404 error for a method you're supposed to be able to call usually indicates a misconfiguration or permissions issue. Since other APIs work and CatalogServiceClient works, we can likely rule out basic credential problems.

Here are some approaches that you may try:

  1. Dataplex is a regional service. You must specify the region when creating the DataplexServiceClient. The default endpoint might not be correctly configured or the operations are not available in the default region. If you don't specify the correct region, it can lead to 404 errors.

Double-check the region where your Dataplex resources (lakes, zones, assets) are created. Use the same region in your code. The error message "Received http2 header with status: 404" strongly suggests the client is looking in the wrong region. You can also check which regions Dataplex is available in, in case you are using an unsupported region.

  1. While you said other APIs work, double-check the service account or user account you're using has the necessary Dataplex permissions.
  • Required Permissions: To use DataplexServiceClient, your account needs roles like:
    * roles/dataplex.admin: Full access to Dataplex resources.
    * roles/dataplex.viewer: Read-only access to Dataplex resources.
    * roles/dataplex.metadataReader: Read metadata.
    * roles/dataplex.dataScanAdmin, roles/dataplex.dataQualityScanAdmin, etc. (for specific features)
  • Check IAM: Go to the IAM & Admin section of the Google Cloud Console, find the service account or user you're using, and verify it has the appropriate Dataplex roles. Grant the roles/dataplex.admin role for testing purposes to eliminate permissions issues.
  • If running in a Compute Engine instance or other managed service, ensure the instance's service account has the required roles.

  1. Ensure you have the latest version of the google-cloud-dataplex library. Sometimes older versions have bugs or compatibility issues.

  2. If you're running your code within a restricted network (e.g., behind a firewall or using VPC Service Controls), ensure that your network allows outbound traffic to the Dataplex API endpoint (usually *-dataplex.googleapis.com).

VPC Service Controls can restrict access to specific regions, which could cause the 404 if the Dataplex service isn't accessible in the allowed region.

  1. Ensure the project ID you're using is the correct project where your Dataplex resources are created.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

0 Likes

Posted on --/--/---- --:-- AM
danielc_68
Bronze 1
Bronze 1
In response to ruthseki
Reply posted on --/--/---- --:-- AM

Thank you for reply,

First suggestion looked promising, since I'm using default configuration that points to dataplex.googleapis.com. But are you sure that it's in fact a regional service ? dataplex api docs point to the global endpoint I'm currently using https://cloud.google.com/dataplex/docs/reference/rest/. Anyway I tried to overwrite it via

options = ClientOptions(
    api_endpoint="https://dataplex.us-central1.rep.googleapis.com/",
)
client = dataplex_v1.DataplexServiceAsyncClient(client_options=options)

but that one failed with

google.api_core.exceptions.RetryError: Timeout of 60.0s exceeded, last exception: 503 DNS resolution failed for https://dataplex.us-central1.rep.googleapis.com/: C-ares status is not ARES_SUCCESS qtype=A name=https is_balancer=0: Domain name not found

 

That might suggest some network problem on my side, I will try to do the same from a VM. 

IAM part should be fine, I'm owner in that project and checked that I can use dataplex from console.  

0 Likes