How to get variables from the Secret Manager and apply them to the build of my app.yaml file ?

[whostris]

Hey,

I have a Android/iOS application and I migrated my services to Google cloud (SQL, Storage and App Engine).

All works but I would like to set up a CI/CD system so that when we apply changes to the node.js API and push to /main on the github repo, gcloud automatically does the build. gcloud app deploy is good but not practical. So I configured 2 files at the root of the node.js project: app.yaml which contains the environment variables necessary for my API to works and cloudbuild.yaml which contains the instructions for Cloud build

The problem is that I cannot retrieve the variables from the Secret Manager, the API does not keep them during runtime. Only during the buildtime I have the impression. The other problem is that I don't want the app.yaml file to end up on Github with variables visible. So either if I put app.yaml in the .gitignore the build fails because gcloud doesn't find the file, or I push it to github trying to retrieve the env variables but it doesn't work

If you have an idea, I'm lost in this new environment. Thanks in advance

# app.yaml
runtime: nodejs18
instance_class: F1

env_variables:
    HOST: "0.0.0.0"
    NODE_ENV: "production"    
    DATABASE_NAME: ${_DATABASE_NAME}
    DATABASE_USER: ${_DATABASE_USER}
    DATABASE_PASSWORD: ${_DATABASE_PASSWORD}
    INSTANCE_CONNECTION_NAME: ${_INSTANCE_CONNECTION_NAME}
    GCS_BUCKET_NAME: ${_GCS_BUCKET_NAME}
    GCS_BASE_PATH: ${_GCS_BASE_PATH}

beta_settings:
    cloud_sql_instances: ${_INSTANCE_CONNECTION_NAME}

# cloudbuild.yaml
steps:
    - name: "node:18"
      entrypoint: npm
      args: ["install"]

    - name: "node:18"
      entrypoint: npm
      args: ["run", "gcp-build"]
      env:
          - "HOST=0.0.0.0"
          - "NODE_ENV=production"
          - "DATABASE_NAME=$_DATABASE_NAME"
          - "DATABASE_USER=$_DATABASE_USER"
          - "DATABASE_PASSWORD=$_DATABASE_PASSWORD"
          - "INSTANCE_CONNECTION_NAME=$_INSTANCE_CONNECTION_NAME"
          - "GCS_BUCKET_NAME=$_GCS_BUCKET_NAME"
          - "GCS_BASE_PATH=$_GCS_BASE_PATH"

    - name: "gcr.io/cloud-builders/gcloud"
      args: ["app", "deploy", "app.yaml"]

timeout: "1800s"
options:
    logging: CLOUD_LOGGING_ONLY

[greb]

Hi @whostris,

Welcome to Google Cloud Community!

I understand that your main concerns here are:  safeguarding secrets and making sure your deployment operates seamlessly with CI/CD. Here’s what you can do that may help resolve it:

1. Keep Secrets Secure: Avoid hardcoding sensitive info in app.yaml. Instead, use Cloud Build substitutions to inject secrets from Secret Manager during the build process. For runtime, update your Node.js app to fetch secrets dynamically from Secret Manager using their API.
2. Handle app.yaml Securely: Don’t push it to GitHub. Store it in a secure location like Cloud Storage and add a step in your cloudbuild.yaml to pull it before deployment.
3. Streamline Deployment: With the help of the above tips, your CI/CD setup will deploy without exposing secrets or requiring manual intervention.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.