Google Infrastructure Manager - Error on terraform_show step preventing infrastructure preview

[Masagero]

I have been trying to deploy GCP infrastructure (e.g. Google Cloud Storage) with Google Infrastructure Manager.
Regarding https://cloud.google.com/infrastructure-manager/docs/automate-git I have been trying to automate deployments of Google Cloud resources (e.g. Google Cloud Storage) using my existing GitHub repository and Google Infrastructure Manager.
As I noticed, Infrastructure Manager creates 4 builds if we have 2 triggers - on create pull request and on merge pull request.
Here is what I achieved (almost everything is working):
When I create a pull request to the GitHub repository, Infra Manager creates a "preview process build" which is responsible for creating "preview build" which creates execution plan for infrastructure.
The "preview build" ends successfuly, but the "preview process build" itself gives errors on terraform_show step, so can't see the plan what is going to be provisioned.
Anyway, when I merge a pull request into the GitHub repository, Infra Manager creates a "apply process build" which is responsible for creating a "apply build" which deploys Google Cloud resoruces.
Both "apply process build" and "apply build" end successfuly, so resources are properly deployed.

The error which I get is:
starting build "b569a3dd-87a1-4837-98ef-249f909896ac"
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/user_anonymized/repo_anonymized
* branch 7ccf896b7243493f96c250699327a3ad2bc76203 -> FETCH_HEAD
HEAD is now at 7ccf896 asd
GitCommit:
7ccf896b7243493f96c250699327a3ad2bc76203
SETUPBUILD
BUILD
Starting Step #0 - "git_setup"
Already have image (with digest): gcr.io/cloud-builders/git
***** NOTICE *****
Official `cloud-sdk` images, including multiple tagged versions across multiple
platforms, can be found at
https://github.com/GoogleCloudPlatform/cloud-sdk-docker and may be more suitable
for some use cases when interacting with Cloud Source Repositories.
For additional information, please visit
https://github.com/GoogleCloudPlatform/cloud-builders/tree/master/git
***** END OF NOTICE *****
Finished Step #0 - "git_setup"
Starting Step #1 - "create_preview"
Already have image (with digest): gcr.io/cloud-builders/gcloud
Checking if deployment tf-jobapps-create-infrastructure already exists
Deleting previous preview if it already exists
ERROR: (gcloud.infra-manager.previews.delete) NOT_FOUND: Resource 'projects/project_anonymized/locations/us-central1/previews/preview-7ccf896' was not found. This command is authenticated as sa-anonymized@project_anonymized.iam.gserviceaccount.com which is the active account specified by the [core/account] property
- '@type': type.googleapis.com/google.rpc.ResourceInfo
resourceName: projects/project_anonymized/locations/us-central1/previews/preview-7ccf896
Creating the preview...
............................................................................................................................................................................................................................................................................................................................................done.
Created preview: projects/project_anonymized/locations/us-central1/previews/preview-7ccf896
Finished Step #1 - "create_preview"
Starting Step #2 - "download_preview"
Already have image (with digest): gcr.io/cloud-builders/gcloud
Initiating export preview results...
Exported preview artifacts plan.tfplan and plan.json
Finished Step #2 - "download_preview"
Starting Step #3 - "terraform_init"
Pulling image: hashicorp/terraform:1.5.7
7264a8db6415: Already exists
22fdd27f372d: Pulling fs layer
d7274e049972: Pulling fs layer
22fdd27f372d: Verifying Checksum
22fdd27f372d: Download complete
d7274e049972: Verifying Checksum
d7274e049972: Download complete
22fdd27f372d: Pull complete
d7274e049972: Pull complete
Digest: sha256:9fc0d70fb0f858b0af1fadfcf8b7510b1b61e8b35e7a4bb9ff39f7f6568c321d
Status: Downloaded newer image for hashicorp/terraform:1.5.7
docker.io/hashicorp/terraform:1.5.7
Terraform initialized in an empty directory!
The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.
Finished Step #3 - "terraform_init"
Starting Step #4 - "terraform_show"
Already have image (with digest): hashicorp/terraform:1.5.7
Error: Failed to load plugin schemas
Error while loading schemas for plugin components: 3 problems:
- Failed to obtain provider schema: Could not load the schema for provider
registry.terraform.io/hashicorp/google: failed to instantiate provider
"registry.terraform.io/hashicorp/google" to obtain schema: unavailable
provider "registry.terraform.io/hashicorp/google".
- Failed to obtain provider schema: Could not load the schema for provider
registry.terraform.io/hashicorp/google-beta: failed to instantiate provider
"registry.terraform.io/hashicorp/google-beta" to obtain schema: unavailable
provider "registry.terraform.io/hashicorp/google-beta".
- Failed to obtain provider schema: Could not load the schema for provider
registry.terraform.io/hashicorp/random: failed to instantiate provider
"registry.terraform.io/hashicorp/random" to obtain schema: unavailable
provider "registry.terraform.io/hashicorp/random"..
Finished Step #4 - "terraform_show"
ERROR
ERROR: build step 4 "hashicorp/terraform:1.5.7" failed: step exited with non-zero status: 1

I need assistance in resolving the error occurring during the terraform_show step of the "preview process build" so that I can successfully view the infrastructure preview before merging pull requests.

[nmagcalengjr]

Hi @Masagero,

Welcome to Google Cloud Community! 

Here are some basic troubleshooting steps you can follow:

1. Make sure Terraform config files are included in your Cloud Build context:

In your cloudbuild.yaml or Infrastructure Manager configuration, ensure the Terraform .tf files are accessible and properly included so that terraform init and terraform show can execute successfully.

steps:

  - name: gcr.io/cloud-builders/git

    entrypoint: bash

    args:

      - -c

      - |

        git clone https://github.com/your_org/your_repo.git .

  - name: gcr.io/cloud-builders/gcloud

    id: create_preview

    args: ['infra-manager', 'previews', 'create', ...]

  - name: gcr.io/cloud-builders/gcloud

    id: download_preview

    args: ['infra-manager', 'previews', 'export', '--preview=...']

  - name: hashicorp/terraform:1.5.7

    id: terraform_init

    args: ['init']

  - name: hashicorp/terraform:1.5.7

    id: terraform_show

    args: ['show', '-json', 'plan.tfplan']

2. Use Terraform Plugin Cache (optional but helpful):

To persist Terraform providers across builds, use a shared cache or store them in a GCS bucket by configuring your build steps accordingly, ensuring faster and more efficient subsequent executions.

• Clear Terraform cache: Sometimes, a corrupted cache can cause problems. Try clearing the Terraform cache with terraform cache purge.

env:

  - TF_PLUGIN_CACHE_DIR=/wor

To verify the setup, manually run the steps in Cloud Shell or a custom Cloud Build to ensure terraform show plan.tfplan executes correctly, and confirm that terraform init is being run in the correct directory containing your .tf files.

If you need further assistance, you can reach out to Google Cloud Support at any time.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[Masagero]

Hello @nmagcalengjr,
I'm using Google Infrastructure Manager configuration which built for me both Cloud Build Triggers.

im-apply-1234567890-anonymous - here entire process works fine

im-preview-1234567890-anonymous - this process throws an error, but the build with preview itself creates succesfully

Both build triggers have standard inline configuration created in YAML by Google Infrastructure Manager. I didn't change anything manually.

The configuration for im-preview-1234567890-anonymous trigger is:

steps:
  - name: gcr.io/cloud-builders/git
    env:
      - SHORT_SHA=$SHORT_SHA
    args:
      - config
      - '--global'
      - init.defaultBranch
      - main
    id: git_setup
  - name: gcr.io/cloud-builders/gcloud
    env:
      - SHORT_SHA=$SHORT_SHA
    id: create_preview
    script: >
      #!/usr/bin/env bash
      echo "Checking if deployment tf-jobapps-create-infrastructure already
      exists"
      DEPLOYMENT_EXISTS=$(gcloud infra-manager deployments list --location
      us-central1 --filter tf-jobapps-create-infrastructure | tail -n +2 | wc
      -l)
      echo "Deleting previous preview if it already exists"
      gcloud infra-manager previews delete
      projects/im-test-anonymous/locations/us-central1/previews/preview-$SHORT_SHA
      --quiet
      CREATE_PREVIEW_CMD="gcloud infra-manager previews create
      projects/im-test-anonymous/locations/us-central1/previews/preview-$SHORT_SHA
      \
        --service-account=projects/im-test-anonymous/serviceAccounts/sa-infra-test@im-test-anonymous.iam.gserviceaccount.com \
        --git-source-repo=https://github.com/anonymous/anonymous.git \
        --git-source-ref=$SHORT_SHA"
      if [[ "infra/environments/dev" != "" ]]; then
        CREATE_PREVIEW_CMD+=" --git-source-directory=infra/environments/dev"
      fi
      if [[ "" != "" ]]; then
        CREATE_PREVIEW_CMD+=" --input-values="
      fi
      if [[ $DEPLOYMENT_EXISTS -eq 1 ]]; then
        CREATE_PREVIEW_CMD+=" --deployment projects/im-test-anonymous/locations/us-central1/deployments/create-infrastructure"
      fi
      $CREATE_PREVIEW_CMD
      if [[ $(echo $?) -ne 0 ]]; then
        gcloud infra-manager previews describe projects/im-test-anonymous/locations/us-central1/previews/preview-$SHORT_SHA
        exit 1
      else
        exit 0
      fi
  - name: gcr.io/cloud-builders/gcloud
    env:
      - SHORT_SHA=$SHORT_SHA
    args:
      - infra-manager
      - previews
      - export
      - >-
        projects/im-test-anonymous/locations/us-central1/previews/preview-$SHORT_SHA
      - '--file'
      - plan
    id: download_preview
  - name: 'hashicorp/terraform:1.5.7'
    env:
      - SHORT_SHA=$SHORT_SHA
    args:
      - init
      - '-no-color'
    id: terraform_init
  - name: 'hashicorp/terraform:1.5.7'
    env:
      - SHORT_SHA=$SHORT_SHA
    args:
      - show
      - /workspace/plan.tfplan
      - '-no-color'
    id: terraform_show
timeout: 600s
options:
  logging: CLOUD_LOGGING_ONLY

We can see in Step #1 - "create_preview" within the script, that Terraform .tf files in GitHub are pointed out.

Moreover the preview is created and I can see the details and export and download it manualy as binarySignedUri or jsonSignedUri.

gcloud infra-manager previews describe projects/$PROJECT_ID/locations/$REGION/previews/preview-9dc5cb2

gcloud infra-manager previews export projects/$PROJECT_ID/locations/$REGION/previews/preview-9dc5cb2

The Step #2 - "download_preview" is also successful.

But the Step #3 - "terraform_init" and Step #4 - "terraform_show" fail.

The configuration in main.tf file covers Infrastructure Manager module regarding to documentation.

module "infrastructure-manager-workspace" {
    source = "terraform-google-modules/bootstrap/google//modules/im_cloudbuild_workspace"
    version = "~> 11.0"
    project_id = var.project_id
    deployment_id = var.deployment_id
    im_deployment_repo_uri = var.git_repo
    im_deployment_ref = "master"
    im_deployment_repo_dir = var.git_repo_directory
    tf_version = "1.5.7"
    github_pat_secret = var.git_sa_secret
    repo_connection_name = var.tf_conn_name
    infra_manager_sa = var.sa_infra_mgr_full
    location = var.region
    cloudbuild_sa = var.sa_infra_mgr_full
    github_app_installation_id = var.git_app_id
}

What is missing from my configuration?