Request to Metadata service is failed on minikube with Cloud Code for VS Code

[Takamizawa]

Hi,

I'm a beginner of Cloud Run and k8s.
Now I'm developing with Node.js to use Cloud Run and trying to use SecretManager to save secrets.
But, it's never successful because the request to Metadata Service on minikube pods failed.
I think my code is not wrong because the code is able to get the secret from SecretManager when the script is executed on my shell.
I think the problem is with the request to the Metadata Service in the pod, since the request outside the pod is successful.
Does anyone know how to solve this problem?

Error:

Error: Could not refresh access token: request to http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform failed, reason: socket hang up
    at Gaxios._request (/Users/takamizawa/dev/functions/node_modules/gaxios/build/src/gaxios.js:148:19)
    at process.processTicksAndRejections (/Users/takamizawa/dev/functions/lib/internal/process/task_queues.js:95:5)
    at async metadataAccessor (/Users/takamizawa/dev/functions/node_modules/gcp-metadata/build/src/index.js:94:21)
    at async Compute.refreshTokenNoCache (/Users/takamizawa/dev/functions/node_modules/google-auth-library/build/src/auth/computeclient.js:57:20)
    at async Compute.getRequestMetadataAsync (/Users/takamizawa/dev/functions/node_modules/google-auth-library/build/src/auth/oauth2client.js:298:17)
    at async Compute.getRequestHeaders (/workspace/node_modules/google-auth-library/build/src/auth/oauth2client.js:261:26) {config: {…}, response: undefined, error: FetchError, code: 'ECONNRESET', note: 'Exception occurred in retry method that was not classified as transient', …}

Environment:
Cloud Code for VS Code on Mac, Ventura(13.6)
launch.json

{
    "configurations": [
        {
            "name": "Cloud Run: Run/Debug Locally",
            "type": "cloudcode.cloudrun",
            "request": "launch",
            "build": {
                "buildpacks": {
                    "path": "PATH_TO/package.json",
                    "builder": "gcr.io/buildpacks/builder:latest"
                }
            },
            "image": "functions",
            "service": {
                "name": "functions",
                "containerPort": 8080,
                "resources": {
                    "limits": {
                        "memory": "512Mi"
                    }
                }
            },
            "target": {
                "minikube": {}
            },
            "watch": true
        }
    ]
}

package.json

{
    "name": "My Cloud Run",
    "version": "1.0.0",
    "description": "",
    "main": "index.js",
    "scripts": {
       "test": "echo \"Error: no test specified\" && exit 1",
       "start": "node index.js"
     },
    "engines": {
       "node": ">=16.0.0"
     },
    "author": "",
    "license": "ISC",
    "dependencies": {
       "@google-cloud/secret-manager": "^5.0.1",
       "express": "^4.18.2"
     }
}

index.js

const express = require('express');
const bodyParser = require('body-parser');
const crypto = require('crypto');
const {SecretManagerServiceClient} = require('@google-cloud/secret-manager');

const app = express();

app.use(bodyParser.raw({type: 'application/json'}));
app.post('/', async (req, res) => {
    console.log('Request received');
    try {
        const HMAC = req.headers['x-hmac'];
        const isValid = await verifyWebhook(HMAC, req.body);
        if (!isValid) {
            res.status(401).send('Unauthorized');
            return;
        }
        const body = JSON.parse(req.body);
        // some process will be implemented.
        res.send();
    } catch (err) {
        console.error(err);
        res.status(500).send('Internal Server Error');
    }
});

const verifyWebhook = async (HMAC, body) => {
    const secret = await getSecret();
    const hash = crypto.createHmac('sha256', secret).update(body).digest('base64');
    return HMAC === hash;
}

const getSecret = async () => {
    const client = new SecretManagerServiceClient({fallback: true});
    const name = process.env.SECRET_NAME;

    const [secret] = await client.accessSecretVersion({
        name: name,
    },
    {
        timeout: 200000
    });
    return secret.payload.data.toString();
}

const port = parseInt(process.env.PORT) || 8080;
app.listen(port, () => {
    console.log(`Listening on port ${port}`);
});

module.exports = app;

[lawrencenelson]

Hi @Takamizawa,

Welcome to the Google Cloud Community!

Before starting, please ensure you've backed up your application.

To troubleshoot the issue, you might want to try authenticating using a service account. Here's how you can do it:

1. Navigate to your Google Cloud Console.
2. Go to IAM & Admin > Service Accounts.
3. Click on "Create Service Account".
4. Provide a name for the Service account and fill in the Service account ID (required). Optionally, assign roles to the service account, granting it permissions to perform specific actions on resources within your project.
5. After filling in the details, click "Create".
6. Navigate to the service account you've just created and click on its name.
7. Inside the "Keys" section, click on "ADD KEY", then choose Create new key and select the JSON format. This action will prompt a download of a JSON file. It's crucial to keep this file secure, as possession of this key provides access to your GCP resources.
8. Move the newly created .json key to the directory where your launch.json application resides.

Now, update your launch.json file and insert the following code block:

{
  "configurations": [
    {      
      "env": {
        "GOOGLE_APPLICATION_CREDENTIALS": "/home/lawrencenelson/projects" // Replace with the actual path to your service account key.
      },
    }
  ]
}

After updating, try running the application again.

If the application still can't detect the GOOGLE_APPLICATION_CREDENTIALS, consider setting up the environment variable externally. You may refer to this Stack Overflow thread if you ever encounter the issue.

If the above options don't work, you can contact Google Cloud Support to further look into your case. Thank you!