Metadata error - Google App Engine

[ffejrekaburb]

Hi,

I have the following sync service which is creating an error on a meta dataservice. Can I ask for help interpreting this? What is the meta dataservice? Is there a link to this logic?

sync.yml

service: sync

instance_class: F2

automatic_scaling:

  max_instances: 1

runtime: python312

app_engine_apis: true

entrypoint: gunicorn -b :$PORT sync:app

#inbound_services:

#- warmup

#libraries:

#- name: jinja2

#  version: latest

#- name: ssl

#  version: latest

# taskqueue and cron tasks can access admin urls

handlers:

- url: /.*

  script: sync.app

  secure: always

  redirect_http_response_code: 301

env_variables:

  MEMCACHE_USE_CROSS_COMPATIBLE_PROTOCOL: "True"

  NDB_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: "True"

  DEFERRED_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: "True"

  CURRENT_VERSION_TIMESTAMP: "1677721600"

sync.py 

import google.appengine.api

client = ndb.Client()

def ndb_wsgi_middleware(wsgi_app😞

    def middleware(environ, start_response😞

        with client.context():

            return wsgi_app(environ, start_response)

    return middleware

log that appears when service is starting

app.wsgi_app = ndb_wsgi_middleware(google.appengine.api.wrap_wsgi_app(app.wsgi_app))

2024-05-17 04:04:53 sync[20240515t183736] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 373, in _next_batch
2024-05-17 04:04:53 sync[20240515t183736] response = yield _datastore_run_query(query)
2024-05-17 04:04:53 sync[20240515t183736] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-17 04:04:53 sync[20240515t183736] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet
2024-05-17 04:04:53 sync[20240515t183736] yielded = self.generator.throw(type(error), error, traceback)
2024-05-17 04:04:53 sync[20240515t183736] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-17 04:04:53 sync[20240515t183736] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 1030, in _datastore_run_query
2024-05-17 04:04:53 sync[20240515t183736] response = yield _datastore_api.make_call(

2024-05-17 04:04:45 sync[20240515t183736] google.api_core.exceptions.RetryError: Maximum number of 3 retries exceeded while calling <function make_call.<locals>.rpc_call at 0x3e956b7625c0>, last exception: 503 Getting metadata from plugin failed with error: Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable

[juliadeanne]

Hello @ffejrekaburb,

Welcome to the Google Cloud Community!

The Google Compute Engine metadata service provides information about the GCP instance where your application is running. This includes details like project ID, instance ID, service accounts, and network configuration. It looks like your sync.py code is trying to access the default service account using this metadata service.

Error message shows that your application can't retrieve information from the metadata service. Error code 503 means "Service Unavailable." This could be due to:
- Temporary unavailability of the metadata service due to maintenance or other issues.
- Network connectivity problems between your application and the metadata service.
- Lack of authorization for your application to access the metadata service (though this is less likely).

For more help on troubleshooting metadata server access issues, check out the Troubleshooting Metadata Server access issues documentation.

[ffejrekaburb]

Thank you, this helped a lot!

I'm suspecting point 3 is the concern.

For context, this is a google app engine project named sync. How do I enable authorization? My understanding is it defaults to authorized for main.py. I had to branch off with gunicorn [1] to support this service.

-- sync.yaml --

entrypoint: gunicorn -b :$PORT sync:app

[1] - App Engine app.yaml reference  |  Google App Engine standard environment docs  |  Google Cloud

[SP_2023]

Hi @ffejrekaburb could you overcome the challenge of Metadata error? I have been struggling with the same.

[ffejrekaburb]

No, I'm still looking for support. The root for me is another python2 to python3 change.