How to configure hub/spokes egress through a single VPC?

[marioapra]

Hello all, I'm trying to figure out how I can have all the outbound traffic to go through a single vpc.

Requirements:
- all egress traffic goes through a single cloud NAT in a vpc (called hub)
- traffic between two different spokes needs to go throught hub.
- I don't want to change my applications running in the spokes (for example letting them be aware that they are behind a proxy)

I want to have a hub-spoke architecture.  The spokes will be located in a different project (and different vpc).
For all the spokes, the default route that goes to the default internet gateway will be deleted, together with the cloud nat created by default.

The hub and spokes are currently connected to each other via NCC (Network connectivity center), but I'm open to connect them differently (vpc peering, vpn, etc).

My biggest blocker is that when I try to create a route in the spoke VPC and seeting any resource in the hub vpc as a next hop, it doesn't let me.

What I've tried so far:
- create a Secure Web Proxy in the hub and configure the spoke to use the proxy.  The downside of this is that every application in the spoke needs to be aware that it's running behind a proxy, so that's not viable.  For example this is required to work: `curl example.com --proxy private-web-proxy-ip:443`
- creating a VM in the hub (with port foward enabled) that should be used for NAT.  That machine runs "sysctl -w net.ipv4.ip_forward=1 && iptables -t nat -A POSTROUTING -o $(/sbin/ifconfig | head -1 | awk -F: {\'print $1\'}) -j MASQUERADE" and it's running behind a internal load balancer, so it can be auto-scaled easily.  (inspired by https://cloud.google.com/bare-metal/docs/bms-setup#bms-access-internet-vm-nat).  It didn't work for me because I'm still using NCC, so the routes created in the hub are not propagated to the spokes.  When I try to set the vm ip or the load balancer as a next hop, it always complains saying that the range is outside of the network
- My next attempt will be to create a vm in the spoke to talk with the vm in the hub.  resource in spoke that wants internet -> vm in spoke -> vm in hub -> cloud nat -> internet.  This is quite annoying and it will be costly to have one vm per spoke

[ammett]

Checkout these codelabs  and see if any of these may point you in the right direction .
- CodeLab: Dynamic Route Exchange with NCC  
- More NCC codelabs 

[marioapra · Accepted solution]

Turns out NCC doesn't support this yet.

To do an instance-based NAT service (with regular VPC peering), is actually a little bit simpler and that's the guide:

https://cloud.google.com/load-balancing/docs/internal/deploying-ilb-next-hop-vm

In summary: Create an instance group (with public IPs on them) with an internal load balancer in front of it. Then create a static default route *to the load balancer* which then gets installed into the routing table of its own VPC and its peers.

In order to share this route to multiple different regions, then I just need to  enable global access on the internal load balancer for it to be available in multiple regions.

More details here: https://cloud.google.com/load-balancing/docs/internal/setting-up-internal#ilb-global-access

[kingcartman73]

Have you tried Secure Web Proxy in transparent mode?  If using legacy peering, then this is supported.

https://cloud.google.com/secure-web-proxy/docs/deploy-next-hop

It does have some limitations though.

I had the exact same use case as you.  I used PSC service attachment mode for SWP.I only used one endpoint, and propagated the route root it via NCC (Which is now in preview). It works really nicely, but having to configure local proxy settings is a pain, but not the end of the world.

Is it better than having to maintain VMs? I think so personally, but it depends on your situation I guess.

NCC by design don’t support use of custom static routes, so any next hop options for NCC are off the table.

[marioapra]

I tried, I remember that it didn't support setting the proxy as a next hop across different projects.

It's not possible for me to configure the local proxy settings for all applications and ensure that every people who deploys there need to configure the proxy settings.  Also, there are some old applications that might not support the proxy by default which is annoying.

Another limitation was that it only works with HTTP protocol, so if I want to use anything else (example UDP) it just wouldn't work 😞