GCP VM list across GCP organization taking time

[gcloudLearning]

I am trying to list gcp vm name and project id, project name across GCP organization(gcp folders and projects). 

I have used gcloud vm instance , project list and describe command ,which is working fine and giving the data using python script. total its showing 150 vm records as we have multiple folders and project id. I executed the solution in gcloud shell and also using cloud run job. it's taking 30 min to iterate overs all projects id and vm . 

since it's just listing command and total 150 vms . Can you please let me know if there is any solution which takes less time. because 30 min is quite huge.

example - In azure we have resource graph query which completes similar task in 4-5 minutes for entire tenant and provide vm data and subscription id. can we achieve same in GCP or any similar solution which takes less time .

[gcloudLearning · Accepted solution]

can you please suggest how to get these 2 more  fields(project name,  label). project id is coming but not project name. also I have to get one of the label (env:prod). I have to fetch the value of env.

gcloud asset search-all-resources --scope='organizations/{orgid}' --asset-types='compute.googleapis.com/Instance' --read-mask=displayName,project --format="table(displayName,project)" | awk '{gsub("projects/","",$2); print $1 "\t" $2}'

[kumards]

Hi @gcloudLearning ,

Have you considered using Cloud Asset Inventory?

E.g., you can use the following command to get the names of all the Compute Engine VMs in all the folders & projects in an organization:

gcloud asset search-all-resources --scope='organizations/{ORGANIZATION_NUMBER}' --asset-types='compute.googleapis.com/Instance' --read-mask=displayName

I tested this command for an organization that has ~700 VMs. The command took ~10 seconds. 

For more info about this command & others that Cloud Asset Inventory supports, please see https://cloud.google.com/sdk/gcloud/reference/asset/search-all-resources.

I hope this helps.

[gcloudLearning · Accepted solution]

can you please suggest how to get these 2 more  fields(project name,  label). project id is coming but not project name. also I have to get one of the label (env:prod). I have to fetch the value of env.

gcloud asset search-all-resources --scope='organizations/{orgid}' --asset-types='compute.googleapis.com/Instance' --read-mask=displayName,project --format="table(displayName,project)" | awk '{gsub("projects/","",$2); print $1 "\t" $2}'

[kumards · Accepted solution]

Sorry, I missed the project name part that you mentioned in the original question.

• You can get the project number by adding "project" to the read-mask argument, as you did in the command in your previous message.
• To get the labels, add "labels" to the read-mask argument. You can filter the output for just the "env" labels by adding a query argument.

The following command returns the VM names, project numbers, and labels for the VMs that have "env:*" labels:

gcloud asset search-all-resources --scope='organizations/{ORGANIZATION_NUMBER}' --asset-types='compute.googleapis.com/Instance' --read-mask=displayName,project,labels --query=labels.env:*

To get the project name (ID), you would need to take the project number from the output of the preceding command and then run the following command:

gcloud projects describe {PROJECT_NUMBER} --format="value(projectId)"

You mentioned a Python script in your original message. Could you adjust the script to use the above commands? I'm keen to hear how that goes.