Cloud NAT failing for basic use case

[recseiuq]

Hello,

I am using shared VPC with a single network host project containing several VPCs. These VPCs have multiple subnets and in one of the VPCs I set up Cloud NAT some time ago and it has been working well. Today I tried to set up Cloud NAT in another of the VPCs to enable internet access but it doesn't work.

When I run the GCP Connectivity Tester to try to send traffic from one of the GCE instances to an IP address on the public internet, the outcome is as follows:

• VM instance - success.
• Default egress firewall rule - success
• Packet could be dropped | Configuration analysis determines that the packet could be dropped due to no routes

However, just as with the VPC for which the Cloud NAT is working, there is a default route to the internet in this VPC. As might be expected, the test works for a VM behind the working Cloud NAT.

Configuration is via Terraform and the Cloud NAT instances are managed using the terraform-google-cloud-router module (v5.0.0) with default values.

module "cloud_nat_cloudops" {

  source  = "/path/to/terraform-google-cloud-router"

  name    = "sandbox-nat-subnet2"

  project = "my-project-id-123456'

  region  = "australia-southeast-1"

  network = "my-2nd-vpc-network"

  nats = [{

    name = "vpc-subnet2-nat"

  }]

}

This results in the Cloud NAT being created with these options:

High availability: Yes

Source subnets and IP ranges: All subnets' primary and secondary IP ranges

Cloud NAT IP addresses: Auto-allocate

The created Cloud Router instance has little in the way of interesting configuration, being the same as its counterpart used by the working Cloud NAT instance, with the exception of naming.

The default route for the affected VPC is configured as below:

default-route-internet-my-2nd-vpc-network

Description: Internet

Network: my-2nd-vpc-network

Route type: Static

IP version: IPv4

Destination IP address range: 0.0.0.0/0

Priority: 1000

Instance tags: egress-internet

Next hop: Default Internet gateway

If I view this default route and select Show Instances from Service Projects, selecting the service project and the region, no instances are displayed, despite there being 3 eligible instances. For the default route for the VPC having the working Cloud NAT instance, the instances are displayed when I select the service project and region. Both default routes are configured the same, with the exception of the name and VPC.

Things I have tried/verified:

- Deleting and recreating the Cloud NAT and Cloud Router instances

- Changing the VPC network routing mode from global to regional and back

- There are no host firewalls configured on the affected instances

Thanks in advance.