CloudNAT connection logging doesn't show post-NAT outgoing port number?

[jogu]

We are experiencing an issue where a client claims that some of our outgoing HTTPS connections aren't reaching their network, so have enabled cloud NAT logging. However the `nat_port` information in the logging does not seem to contain the after-NAT source port number that was used for the connect to the destination IP - and that information doesn't seem to be anywhere in the log.

e.g. in this example the `nat_port` is 900,  which clearly isn't the actual outgoing port as the documentation says CloudNAT will not use outgoing port numbers less than 1024, and checking the connection on the destination machine shows a different port number - one that is not present anywhere in the log message. Is this a bug in the logging or is there some way to map this port number to the actual one?

{
  "insertId": "17adxbyfg56ttg",
  "jsonPayload": {
...
    "connection": {
      "dest_ip": "18.229.116.174",
      "dest_port": 443,
      "src_port": 57836,
      "src_ip": "10.72.0.13",
      "nat_ip": "35.196.44.185",
      "protocol": 6,
      "nat_port": 900
    }
  },
  "resource": {
    "type": "nat_gateway"
  },
  "timestamp": "2024-02-07T10:27:20.729294750Z",
...
}

[mahmoudrabie]

Hi @jogu 

I would go with the following troubleshooting steps:

(1) Ensure that Cloud NAT logging is correctly configured to log the necessary details, such as translated source IP and port information

Here is the command to enable or verify Cloud NAT logging

gcloud compute routers nats update [NAT_NAME] \
  --router=[ROUTER_NAME] \
  --region=[REGION] \
  --log-filter=ALL

(2) Analyze the Cloud NAT logs to understand the NAT behavior and look for any anomalies in the logging of the NAT port.

gcloud logging read "resource.type=nat_gateway AND jsonPayload.connection.nat_ip=[NAT_IP]" --limit 100 --format json

I hope that helps

Regards

Mahmoud

[Marvin_Lucero]

Hi @jogu ,

If you have access to the destination machine, you can check its logs to see which source port numbers are being used to connect to it. This can be done using the netstat command on the destination machine. You can try this command :

netstat -tunap | grep '18.229.116.174'

It will list all the TCP connections and display them in a human-readable format. The -tunap flag is used to display the TCP connections, UDP connections, and listening ports. It should show an output like the one below:

tcp 0 0 18.229.116.174:443 10.72.0.13:57836 ESTABLISHED

 The src_port field represents the actual source port number that was used for the connection to the destination IP.

So, to map the port number to the actual one, you would need to run this command on the destination machine.