Get all public addresses in one request

[SergeyN]

To ensure the security of the entire organization, I want to collect all public IP addresses across all projects in my organization as quickly and completely as possible.

At the moment, in order to collect all the whitelisted IP addresses of an organization, I have to loop through each project and request the following information:
gcloud compute instances list
gcloud compute addresses list
gcloud sql instances list
gcloud container clusters list

Then filter the server and white IP addresses by mask.

Is there a better way to do this? As far as I know, "Security Command Center" can detect "Asset" with Resource Type "Address" where there are all links to resources with public addresses. How can this be easily and quickly uploaded via the API?

[leoy]

@SergeyN , please post your question to the profiled forum. As far as I know VM public IPs and GKE public IPs can be collected from the list of VPC static and ephemeral IPs. Unfortunately, getting CloudSQL public IP addresses requires a separate call. Needless to say that exposing SQL database via public IP is considered security anti-pattern.

[SergeyN]

@leoy, thanks, but you didn't read my question carefully. I know how to make many different individual calls, but I expected GCP to provide a convenient mechanism for downloading all public addresses across all projects in an organization. And this request will be concise and fast.
At least there is something similar in the "Security Command Center".

[leoy]

Hi @SergeyN , there is indeed no single method to get all public IP addresses that the organization uses for all its services. Some of the services such as AppEngine or CloudRun can be exposed only via GCLB. The CLI commands that you listed do not let you effectively query the whole organization since they are scoped to the single project.

As a mitigation I would like to propose using Asset Manager. The mentioned "Security Command Center" also uses the Asset Manager service to get the information. The information in the service is not updated at real time but presents snapshots made during a day. To get all external (ephemeral and static) IP addresses in your organization can be done using the following Cloud CLI command:

gcloud asset search-all-resources \
  --scope=organizations/{org-id} \
  --asset-types='compute.googleapis.com/Address' \
  --read-mask='*' \
  --format=json

The output json should display all public IP addresses for organization with organization number {org-id}.

Search for entries with .versionedResources[].resource.addressType=="EXTERNAL" .

To run this command you will have to enable cloudasset.googleapis.com API in your organization.

[SergeyN]

@leoy Thanks, but for some reason the "Security Command Center" in the "FINDINGS" section and the "PUBLIC_IP_ADDRESS" category found more addresses than just the Asset mechanism. 

87 vs 46.

Here's how I counted:

gcloud asset search-all-resources \                                                                                                                                                   
  --scope=organizations/XX  \
  --asset-types='compute.googleapis.com/Address' \
  --read-mask='*' \
  --format=json | grep "EXTERNAL" | wc

[SergeyN]

@leoy  Thanks, but for some reason the "Security Command Center" in the "FINDINGS" section and the "PUBLIC_IP_ADDRESS" category found more addresses than just the Asset mechanism.

As you can see in the screenshots 87 vs 46.

[leoy]

The command I provided searches only for public IPs of the VM instances. Public IPs can be exposed by GCLB (Google Cloud Load Balancers) and by CloudSQL instances. Asset inventory supports many types of assets. I don't have an option to experiment right now, but you can run the same command for each asset type or to run it once for multiple asset types and then to analyse the output by querying the resulted json.

For example for Cloud SQL instances, the asset type will be sqladmin.googleapis.com/Instance and each instance will expose an array of IPs in the ipAddresses field.

[JohnnyW]

Is it possible to filter this request to a particular external IP address?  I am really trying to find the best way to determine the project_id of a particular external IP address while pulling the least amount of data.

[leoy]

The returned JSON contains the project id as a "project" field. The project id is also a part of the fully formed name of the IP address resource. There are many methods that you can try to get JSON for "only" your IP. You can filter using "--query" parameter or to minimize total number of returned fields using "--mask" parameter and "grep" your IP with +/- few lines from the output. You can try to use "--filter" parameter just to limit the output to what you need.

[JohnnyW]

Thanks, Leoy.  Seems like something like the following works great:
gcloud asset search-all-resources \
--scope=organizations/XX\
--asset-types='compute.googleapis.com/Address' \
--read-mask=parentFullResourceName,project \
--format=json \
--query="additionalAttributes.address=1.1.1.1"

We use a docker container with google-api-python-client installed and prefer to use that.  Do you know if there is a module in there that can be used to access the same "asset search-all-resources" API?

[JohnnyW]

Actually found that this works well: https://googleapis.github.io/google-api-python-client/docs/dyn/cloudasset_v1.v1.html#searchAllResour...

[leoy]

Yes, you are correct. @glen_yu posted their solution down the thread that uses auto-generated API client in python. You can find more info about this library in Github.

[glen_yu]

Joining this thread late, but funny enough you can't find all the IPs in GCP as easily as you might expect.  So I created this.  It's not 100% but I think it's pretty close.  I do use Cloud Assets API here as well and it returns all IPs internal & external.  See if this helps your use case.