CI/CD with terraform

Kreg · 02-12-2024 08:38 AM

Hi,

I am learning from Cloud Skills Boost how to set up CI/CD in GCP with Cloud Repository using UI or gcloud commands. Now, I am wondering if I can set up whole CI/CD process with Terraform. I mean all commands which I typed to make it complete like: `gcloud artifacts repositories create xxx`, `gcloud container clusters create xxx`, `gcloud source repos create xxx`, `create trigger` or `export some variables` like:

export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
export REGION=us-west1
gcloud config set compute/region $REGION

I know that we have Terraform documentation with specific resources to use https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_rep... however maybe there is website with template which I could use in my case?

Question for more experienced colleagues, up to what point is Terraform useful and at which moment do I need to switch to the GCP console to configure some things?

Thanks in advance!

lawrencenelson

Greetings @Kreg,

It's good that you're eager to explore beyond Cloud Skills Boost and are interested in learning how to automate the entire process using Terraform.

@Kreg wrote:

Now, I am wondering if I can set up whole CI/CD process with Terraform.

@Kreg wrote:

however maybe there is website with template which I could use in my case?

I believe this official GCP GitHub repository answers your question - Secure CI/CD pipeline. This repository offers Terraform modules and sample setups for Google Cloud users to easily set up a secure CI/CD pipeline, following recommendations from the Shifting Left on Security report. These modules present a well-considered architecture, highlighting secure application delivery best practices.

You can also check the official tutorial by HashiCorp here - Get Started - Google Cloud.

@Kreg wrote:

up to what point is Terraform useful and at which moment do I need to switch to the GCP console to configure some things?

Generally, one of the qualities of a good DevOps is the automation of tasks wherever possible, and Terraform is an excellent tool for achieving this. However, there are certain scenarios where using the GCP console may be more advantageous due to some limitations of automation:

Debugging and Troubleshooting: For tasks such as examining logs, reviewing resource specifics, or addressing particular problems, might require the console's GUI.
Custom Configurations or Experimental Features: Terraform might not immediately support the latest features or configurations that demand manual setups, making the console a necessary tool for these instances.
Security-Sensitive Operations: Managing critical security tasks, including the handling of IAM permissions, is often best done directly through the console to ensure more precise control.

I hope this helps. Thank you.

DamianS

Hello @Kreg

The best way to learn is make your hands dirty. You can combine Terraform with gcloud commands ( or even scripts ) to be triggered during deployment by using "null_resource" and provisioner "local-exec" or "remote-exec" depends from which level you are execute terraform apply command. For example I'm deploying my solution directly from gcloud shell ( at organization level ), and I'm using null_resources for exporting mandatory information like in this case PROJECT_ID. Like @lawrencenelson mentioned, official GCP repo is good start, however there are many options to configure CI/CD ( like Github repo, WIF, github actions, GCP org).

resource "null_resource" "set_project" {
  provisioner "local-exec" {
    command = "CLOUDSDK_CORE_PROJECT=$(gcloud config set project $(gcloud projects list --format='value(PROJECT_ID)' --filter='NAME: support-jit'))"
  }
  depends_on = [google_project_service.service]
}
############################
#Make executable
resource "null_resource" "make_executable_folder" {
  count = var.access_at_folder_level ? 1 : 0
  provisioner "local-exec" {
    working_dir = "${path.module}/scripts"
    command     = "chmod a+x deploy_jit_folder_level.sh"
    #interpreter = ["/bin/bash", "-c"]
  }
  #depends_on = [google_project_service.service]
  depends_on = [
    null_resource.set_project
  ]
}

Regarding your question about when you must use GCP itself to configure something. Well, for sure, If you want to create full CI/CD or any kind of automation, shared project with dedicated ServiceAccount and Workload Identity Federation ( or less secured Service Key) will be mostly the case to create it manually. Rest your work can be automated by terraform or bash scripts. Additionally, you should use UI for Organization policy constraint set, as I'm assuming that your Organization is not managed by any IaaC.

Lastly, for learning purposes ( I did it at the beginning ) configure CI/CD by UI and try to rewrite it into terraform , piece by piece. If you don't know how particular resource should be written, you can use https://cloud.google.com/docs/terraform/resource-management/export to export already created infra into terraform state. It will not map your solution 1:1, but it might be some kind of guidance for further work with GCP and Terraform 🙂 I hope, that this will help you to start you work.

cheers,
DamianS