Network and Routing Configuration to manage APIs h...

joel_gauci · ‎10-19-2021

Reminder

In this two-part article (co-authored with Ibrahima N'doye and Christophe Lalevee), we discuss the network and routing configuration for using Apigee X to manage APIs that are hosted on-premise.

As mentioned in the first article of the series, the scope is a situation where the client applications and backend APIs only exist in the on-prem data center and VPC.

We go on with the configuration of the Managed Instance Group (MIG) and HTTPS Internal Load Balancer (ILB)

MIG/HTTPS ILB deployment

In this section, we provide the gcloud commands that can be used to deploy the MIG and HTTPS ILB.

Managed Instance Group

In order to install a MIG, we first need to configure a Compute Engine instance template. Virtual machines based on this template do not expose external IPs (--network-interface=no-address😞

gcloud compute instance-templates create $MIG_NAME \
--project=$PROJECT_ID \
--region=$REGION \
--network=$VPC_NAME \
--subnet=$VPC_SUBNET \
--no-address \
--tags=https-server,apigee-mig-proxy,gke-apigee-proxy \
--machine-type=e2-medium \
--image-family=debian-10  \
--image-project=debian-cloud \
--boot-disk-size=20GB  \
--metadata ENDPOINT=$APIGEE_ENDPOINT,startup-script-url=gs://apigee-5g-saas/apigee-envoy-proxy-release/latest/conf/startup-script.sh

Then we can create the MIG based on the template, define the autoscaling configuration and named ports:

gcloud compute instance-groups managed create $MIG_NAME  \
--project $PROJECT_ID \
--base-instance-name apigee-mig \
--size 2 \
--template $MIG_NAME \
--region $REGION

gcloud compute instance-groups managed set-autoscaling $MIG_NAME \
--project $PROJECT_ID \
--region $REGION \
--max-num-replicas 20  \
--target-cpu-utilization 0.75 \
--cool-down-period 90

gcloud compute instance-groups managed set-named-ports $MIG_NAME  \
--project $PROJECT_ID \
--region $REGION \
--named-ports https:443

You can check that VM instances have been created and that they do not expose an external IP:

gcloud compute instances list --filter=apigee-mig

Here is an example of output:

NAME: apigee-mig-s2ws
ZONE: europe-west1-c
MACHINE_TYPE: e2-medium
PREEMPTIBLE:
INTERNAL_IP: 10.132.0.34
EXTERNAL_IP:
STATUS: RUNNING

NAME: apigee-mig-z6lk
ZONE: europe-west1-d
MACHINE_TYPE: e2-medium
PREEMPTIBLE:
INTERNAL_IP: 10.132.0.33
EXTERNAL_IP:
STATUS: RUNNING

You can now configure the HTTPS ILB, starting with the reservation of a regional IP address.

HTTPS Internal Load Balancer

gcloud compute addresses create lb-ipv4-vip \
--project $PROJECT_ID  \
--ip-version=IPV4  \
--subnet=$VPC_SUBNET \
--region=$REGION

In order to get the value of the reserved IP address, you can execute the following command:

gcloud compute addresses describe lb-ipv4-vip \
--project $PROJECT_ID \
--format="get(address)" \
--region=$REGION

You can use this IP address to create A records in your on-prem DNS server. Hostnames that are defined can be used by client applications and must also be used to define environment groups in your Apigee X organization.

A firewall rule is necessary to allow the internal load balancer to connect with the proxy instances managed by the MIG:

gcloud compute firewall-rules create k8s-allow-lb-to-apigee-mig \
--description "Allow incoming from GLB on TCP port 443 to Apigee Proxy" \
--project $PROJECT_ID \
--network $VPC_NAME \
--allow=tcp:443  \
--source-ranges=130.211.0.0/22,35.191.0.0/16 \
--target-tags=gke-apigee-proxy

Health check used to test the proxy (MIG) instances:

gcloud compute health-checks create https hc-apigee-mig-443 \
--project $PROJECT_ID \
--port 443 \
--region=$REGION   \
--request-path /healthz/ingress

Backend services of the HTTPS internal load balancer (ILB):

gcloud compute backend-services create apigee-mig-backend  \
--project $PROJECT_ID \
--protocol HTTPS \
--load-balancing-scheme=INTERNAL_MANAGED \
--health-checks hc-apigee-mig-443 \
--port-name https \
--timeout 60s \
--connection-draining-timeout 300s \
--health-checks-region=$REGION

Add the backend services to to the HTTPS ILB:

gcloud compute backend-services add-backend apigee-mig-backend \
--project $PROJECT_ID \
--instance-group $MIG_NAME \
--instance-group-region $REGION \
--balancing-mode UTILIZATION \
--max-utilization 0.8 \
--region $REGION

Define the URL map of the HTTPS ILB:

gcloud compute url-maps create apigee-mig-proxy-map \
--project $PROJECT_ID \
--default-service apigee-mig-backend \
--region=$REGION

Set the SSL certificate of the HTTPS ILB. Please note that SSL certificates cannot be managed on an HTTPS ILB:

gcloud compute ssl-certificates create apigee-ssl-cert \
--description="apigee x ssl cert" \
--certificate=./certificate.pem \
--private-key=./key.pem \
--region=$REGION

Create the target HTTPS proxy of the HTTPS ILB, based on the URL map and SSL certificate:

gcloud compute target-https-proxies create apigee-mig-https-proxy \
--project $PROJECT_ID \
--url-map apigee-mig-proxy-map \
--ssl-certificates apigee-ssl-cert \
--region=$REGION

You must create a proxy-only subnet before you create forwarding rules for your internal load balancer. Each region of a VPC network in which you use regional load balancers must have a proxy-only subnet.

If you try to configure a regional load balancer without first creating a proxy-only subnet for the region, the load balancer creation process fails…

Each of the load balancer's proxies is assigned an internal IP address. The proxies for all regional load balancers in a region use internal IP addresses from a single, proxy-only subnet in that region of your VPC network.

A proxy-only subnet must provide 64 or more IP addresses. This corresponds to a prefix length of /26 or shorter

gcloud compute networks subnets create proxy-subnet \
--purpose=INTERNAL_HTTPS_LOAD_BALANCER \
--role=ACTIVE \
--region=$REGION \
--network=$VPC_NAME \
--range=$CIDR_RANGE_PROXY_SUBNET

You must define the value of the CIDR_RANGE_PROXY_SUBNET variable. For example: 10.129.0.0/23 for 512 proxy-only addresses.

Now you can create the forwarding rules of the HTTPS ILB:

gcloud compute forwarding-rules create apigee-mig-https-ilb-rule \
--load-balancing-scheme=INTERNAL_MANAGED \
--network=$VPC_NAME \
--subnet=$VPC_SUBNET \
--address=lb-ipv4-vip \
--ports=443 \
--region=$REGION \
--target-https-proxy=apigee-mig-https-proxy \
--target-https-proxy-region=$REGION

At this step, you need to create a firewall rule in order to allow client applications to connect to the HTTPS ILB on port 443.

gcloud compute firewall-rules create allow-https-ilb \
--direction=INGRESS \
--priority=1000 \
--network=$VPC_NAME \
--action=ALLOW \
--rules=tcp:443 \
--source-ranges=0.0.0.0/0

The source ranges may be modified to only allow specific client applications. Tags can also be used to identify allowed sources.

Your Apigee X runtime is now accessible using an HTTPS ILB on port 443.

Our recommendation here is to associate a hostname to the IP address of the HTTPS ILB, then you can register this hostname in an existing environment group of your Apigee X organization.

You can do this using Apigee APIs.

Conclusion

There are several reasons to isolate your Apigee X managed runtime from Internet access, the primary one being security.

For many financial, government, retail and similar institutions this is a must to benefit from an API Management Platform, which is fully managed and used internally as if it were located on their on-prem network.

Google Apigee reconciles now the best of both worlds: a managed API Management solution for totally private access.

dchiesa1 · ‎12-10-2021

FYI, Note the name on the URL for the startup script:

startup-script-url=gs://apigee-5g-saas/apigee-envoy-proxy-release/latest/conf/startup-script.sh

This suggests that the startup script will run an envoy proxy. This is not actulaly the case. This startup script uses an image with only IPtables rules.