Listing Apigee X deployed proxies that have no traffic

[kurtkanaskie]

Given the proxy deployment limits of 50 per environment and 4250 per organization, it would be nice if Apigee X provided a dashboard or report to show proxies without traffic over a period of time.

Currently, you could use the proxies list UI page to see traffic in the last 24 hours for each proxy or run reports to see active proxies over a time period, but there is no easy way to see proxies that are deployed and unused over a period of time (e.g. 90 days).

Until Apigee X provides this view, an interim solution is to use Apigee APIs to gather this information. This is one of the reasons I Love APIs, it takes just 3 APIs to do this and some bash scripting:

1. List the deployed proxies across all environments
2. List the environments in the organization
3. For each environment get the stats for all the APIs

Then, test to see if each of the deployed APIs have traffic or not and output the results.

Here's the example bash script (only tested on Mac):

#! /bin/bash

# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Help()
{
   # Display Help
   echo
   echo "Used for Apigee X"
   echo "Reports API proxy names that are deployed but have no traffic in the last specified days."
   echo "Uses environment variables for ORG and TOKEN if set."
   echo
   echo "Syntax: $0 [-h|-o|-t|-d"
   echo "options:"
   echo "    -h    Print this help"
   echo "    -o    Orgnaization name (required)"
   echo "    -t    Access token (required)"
   echo "    -d    Days to look back (optional, default and max is 90)"
   echo
   echo "Usage: ./deployed-proxies-with-no-traffic.sh -o x-org-name -d 30 -t $(gcloud auth print-access-token)"
   echo
}

APIGEE_API=apigee.googleapis.com
DAYS=90

while getopts "o:t:d:h" flag
do
	case "${flag}" in
		h) Help
			exit;;
		o) ORG=${OPTARG};;
		t) TOKEN=${OPTARG};;
		d) DAYS=${OPTARG};;
			\?) # Invalid option
			echo "Error: Invalid option"
			Help
		exit;;
	esac
done

if [ "${ORG}" == "" ]; then
	echo "ERROR: Organization is required."; Help; exit
fi

if [ "${TOKEN}" == "" ]; then
	echo "ERROR: Token is required."; Help; exit
fi

if [ "${DAYS}" -gt 90 ]; then
	echo "ERROR: Maximum days look back is 90."; Help; exit
fi

DATE_NOW=$(date +"%m/%d/%Y")
DATE_MINUS_DAYS=$(date -j -v-"${DAYS}"d +"%m/%d/%Y")
AUTH="Authorization: Bearer ${TOKEN}"
TR="${DATE_MINUS_DAYS}%2000:00~${DATE_NOW}%2023:59"
echo "Time Range: ${TR}"; echo

# Get the organization to verify a valid token
HTTP_RESPONSE_CODE=$(curl --write-out '%{http_code}' --output /dev/null -s -H "${AUTH}" "https://${APIGEE_API}/v1/organizations/$ORG")
if [[ "${HTTP_RESPONSE_CODE}" -ge 400 ]]; then
	RED='\033[0;31m'
	NO_COLOR='\033[0m'
	echo -e "${RED}ERROR: GET https://${APIGEE_API}/v1/organizations/${ORG} returned ${HTTP_RESPONSE_CODE} ${NO_COLOR}"
	Help
	exit
fi

# Get all deployed APIs across all environments
DEPLOYED_APIS=$(curl -s -H "${AUTH}" "https://${APIGEE_API}/v1/organizations/${ORG}/deployments" | jq -r .deployments[].apiProxy | sort --unique)
# shellcheck disable=SC2206
DEPLOYED_APIS_ARRAY=(${DEPLOYED_APIS})

# Get list of environments
ENVS=$(curl -s -H "${AUTH}" "https://${APIGEE_API}/v1/organizations/${ORG}/environments" | jq -r .[])

# Get APIs that have traffic for each environment
for E in ${ENVS}; do
	ENV_DIMENSIONS=$(curl -s -H "${AUTH}" "https://${APIGEE_API}/v1/organizations/${ORG}/environments/${E}/stats/apiproxy?select=sum(message_count)&timeUnit=month&timeRange=${TR}" | jq -r '.environments[].dimensions')
	ENV_DIMENSIONS_COUNT=$(echo "${ENV_DIMENSIONS}" | jq length)
	echo "${ENV_DIMENSIONS_COUNT} APIs have traffic in ${E}"
	if [ "${ENV_DIMENSIONS_COUNT}" -ne 0 ]
	then
		ENV_TRAFFIC_APIS=$(echo "${ENV_DIMENSIONS}" | jq -r .[].name | sort)
		# shellcheck disable=SC2206
		ENV_TRAFFIC_APIS_ARRAY=(${ENV_TRAFFIC_APIS})
		for API in "${ENV_TRAFFIC_APIS_ARRAY[@]}"
		do
			echo "    $API"
		done
	fi
		
	ORG_TRAFFIC_APIS="${ORG_TRAFFIC_APIS} ${ENV_TRAFFIC_APIS}"
done

echo; echo ================================================
echo "${#DEPLOYED_APIS_ARRAY[@]} APIs are deployed across all environments"
echo; echo ================================================
echo "APIs that have traffic in last ${DAYS} days."
APIS_WITH_TRAFFIC_ARRAY=()
for API in "${DEPLOYED_APIS_ARRAY[@]}"
do
	# If it's deployed and in the list of APIs with traffic
	if [[ "${ORG_TRAFFIC_APIS}" == *"${API}"* ]]
	then
		APIS_WITH_TRAFFIC_ARRAY+=("${API}")
  		echo "    $API"
	fi
done
echo "${#APIS_WITH_TRAFFIC_ARRAY[@]} APIs have traffic in last ${DAYS} days."


echo; echo ================================================
echo "APIs that are deployed but have no traffic in last ${DAYS} days."
APIS_WITHOUT_TRAFFIC_ARRAY=()
for API in "${DEPLOYED_APIS_ARRAY[@]}"
do
	# If its deployed but not in the list of APIs with traffic
	if [[ "${ORG_TRAFFIC_APIS}" != *"${API}"* ]]
	then
		APIS_WITHOUT_TRAFFIC_ARRAY+=("${API}")
  		echo "    $API"
	fi
done
echo "${#APIS_WITHOUT_TRAFFIC_ARRAY[@]} APIs are deployed but have no traffic in last ${DAYS} days."
echo

Here's a sample output:

$ ./deployed-proxies-with-no-traffic.sh
Time Range: 06/30/2023%2000:00~09/28/2023%2023:59

3 APIs have traffic in dev
    app-engine-url-mask
    catch-all-v1
    pingstatus-v1
6 APIs have traffic in prod
    catch-all-v1
    notarget
    pingstatus-mint-v1
    pingstatus-v1
    sample-mtls
    test
7 APIs have traffic in test
    catch-all-v1
    cloud-run-v1
    identity-facade-v1
    logging-test
    mint-jokes-v1
    mint-riddles-v1
    notarget

================================================
21 APIs are deployed across all environments

================================================
APIs that have traffic in last 90 days.
    app-engine-url-mask
    catch-all-v1
    cloud-run-v1
    identity-facade-v1
    logging-test
    mint-jokes-v1
    mint-riddles-v1
    notarget
    pingstatus-mint-v1
    pingstatus-v1
    sample-mtls
    test
12 APIs have traffic in last 90 days.

================================================
APIs that are deployed but have no traffic in last 90 days.
    access-control
    advapisec-accuweather
    advapisec-hello-world
    advapisec-hello-world-oauth
    advapisec-httpbin
    advapisec-oauth
    local-us-east1
    local-us-west1
    persons-bq-sa-v1
9 APIs are deployed but have no traffic in last 90 days.

Hope this is useful!