VM iptables and VPC IP address MySQL connection troubles

[Raphael_de28]

I'm trying to do MySQL queries from a Cloud Functions script to a Compute Engine CentOS instance via Direct VPC Egress which is configured in Cloud Run. However, the MySQL connection fails on a pre-established server while succeeding on a fresh, (internally) un-configured VM, hence my suspicion on iptables causing this blockage.

So, the pre-established VM has the following configurations (excerpt):

Chain INPUT
ACCEPT tcp -- 10.149.0.0/24 anywhere tcp dpt:mysql
DROP all -- anywhere anywhere 

The 10.149.0.0/24 is the VPC subnet's internal IP range of the Cloud Function (-managed Cloud Run) instance

while the Cloud Function seems to be sending packets, sniffed by tcpdump on the new server) like the following

09:02:26.535414 IP (tos 0x0, ttl 64, id 31774, offset 0, flags [none], proto TCP (6), length 52)
10.149.0.20.38179 > test-mysql-server-2.[redacted]: Flags [.], cksum 0x1e27 (correct), ack 4349, win 16383, options [nop,nop,TS val 402447111 ecr 1331970543], length 0

Tangentially, I have also tried using Java 21's InetAddress.isReachable to check for server health by pinging, but tcpdump never detects any ICMP packets, and ICMP iptables rules don't seem to do anything. Maybe this is relevant, maybe is it not. I am a total novice when it comes to *nix and networking stuff, so please forgive my ignorance on such matters.

Does anyone know the reason for this and care to provide me with some pointers to solving it?

[mahmoudrabie]

Hi @Raphael_de28 

Let's write some practical actions and commands to troubleshoot

(1) Set up a VPC Connector in Cloud Run

gcloud run services update SERVICE_NAME --vpc-connector CONNECTOR_NAME

(2)  Update Cloud Function to use the VPC Connector

gcloud functions deploy FUNCTION_NAME --vpc-connector CONNECTOR_NAME --trigger-http

(3) List the current firewall rules to check if there's a rule allowing traffic on MySQL port (3306)

gcloud compute firewall-rules list

(4) (if necessary) Create/Update a firewall rule , to allow MySQL connections

gcloud compute firewall-rules create allow-mysql --direction=INGRESS --priority=1000 --network=VPC_NAME --action=ALLOW --rules=tcp:3306 --source-ranges=10.149.0.0/24

(5) Test connection to the MySQL server using the MySQL client

mysql -h SERVER_IP -u USERNAME -p

(6) Review MySQL server logs for any relevant connection error messages

sudo tail -f /var/log/mysql/error.log

I hope that helps

Regards

Mahmoud