Unable to Test Cloud Function: failed to fetch builder image

[ben_troup_mrkl]

I have been building out a tool using Cloud Run Functions and haven't submitted a new deployment since October 3rd. I was testing out some things last week but did not deploy them. I then came back this week to test again and clicked "test function" and was met with the following error when the testing server was starting up:

[4:18:53 PM] - Preparing function sources ...
[4:18:53 PM] - Building function sources ...
[4:18:53 PM] - Error: Command failed: cd /gcf-working-dir/function-sources && pack build gcf-ui-function --builder gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder --pull-policy if-not-present --env GOOGLE_FUNCTION_TARGET=main --env GOOGLE_FUNCTION_SIGNATURE_TYPE=cloudevent --env GOOGLE_DEVMODE=1 --env GOOGLE_RUNTIME_VERSION=3.11
ERROR: failed to build: failed to fetch builder image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder:latest': image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder:latest' does not exist on the daemon: not found

at checkExecSyncError (node:child_process:826:11)
at execSync (node:child_process:900:15)
at buildFunction (/app/app.runfiles/google3/cloud/console/web/functions/external/testing_server/workers/build_function_worker.js:17:38)
at Object.<anonymous> (/app/app.runfiles/google3/cloud/console/web/functions/external/testing_server/workers/build_function_worker.js:35:131)
at Module._compile (node:internal/modules/cjs/loader:1095:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
at Module.load (node:internal/modules/cjs/loader:975:32)
at Function.Module._load (node:internal/modules/cjs/loader:816:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12)
at MessagePort.<anonymous> (node:internal/main/worker_thread:183:24)
ERROR: failed to build: failed to fetch builder image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder:latest': image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder:latest' does not exist on the daemon: not found

#############################################################
#############################################################

[4:18:54 PM] - Error while building function sources. Please check logs.

Nothing has been changed on my side since last week. I have gone round and round trying to work out what actually needs to be done to resolve this. I checked the function logs as suggested in the error message and the function is running fine when triggered by pub/sub - no errors. I also get the same error when trying to test a brand new Cloud Function using the default code. 

Any avenues to explore here would be appreciated as I've found nothing online that applies to my situation

Thanks

[greb]

Hi @ben_troup_mrkl,

Welcome to Google Cloud Community!

It looks like what you're encountering suggests that the builder image gcr.io/gae-runtimes/buildpacks/google-gae-22/python/builder:latest is either unavailable or misconfigured. Since no changes were made on your side, it might be related to an issue in GCP's underlying infrastructure, such as depreciation or an update to the builder image.

Here are a few workarounds you can try:

• Specify a Different Builder Image: Since the builder image you're using may no longer be available, try specifying a different or updated image manually during deployment. You can use a general-purpose Google Cloud Buildpack builder:
  --builder gcr.io/buildpacks/builder:v1
• Use a Different Python Runtime Version: It's possible that the issue is specific to Python 3.11 or the gae-22 builder. Try switching to a different, more stable runtime version (like Python 3.10 or 3.9) by modifying your environment variables:
  -env GOOGLE_RUNTIME_VERSION=3.10
• Check for Updates: There might be recent changes or deprecations in GCP. Check the release notes.
• Test in a New Environment: You can try deploying your function in a completely new environment with a different runtime and see if the error persists. This might help isolate whether the issue is with the testing environment or something deeper in the deployment configuration.

If none of these workarounds resolve the issue, you may try checking this similar case and check the variables that you might be missing in configuration, or reach out to GCP support to investigate further, as it could be a wider issue with their builder image repositories.

I hope the above information is helpful.

[hhissami]

How would I specify a different image or create a new image in the GCP UI for my Python script in Google Cloud Functions?

[gmike]

I'm getting the same error. It is happening for all Python versions.

No related issues on the release notes.

How can I do it when you say "Specify a Different Builder Image"? Is it a value that I can pass to the Build environment variables? 

I tried to use BUILDER_IMAGE_URL as explained here: https://cloud.google.com/docs/buildpacks/use-a-specific-builder#specifying_the_builder_version_with_... 

but it returned the same error.

[ben_troup_mrkl]

@greb as this seems to be a GCP issue rather than an issue with my configuration, can you offer some alternative next steps? I tried the workarounds you recommended and this didn't resolve the issue, and the article you linked seemed completely unrelated to the issue at hand

Thanks

[jawch]

Im also blocked by this. Please upvote the issue here
https://issuetracker.google.com/issues/376854179

[bradleyjgriffit]

In my case disabling my VPN fixed it. Not sure but worth a try if someone runs into this in the future.