Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Latency to Reach Postgres from GKE/VM

I am currently investigating latency issue being faced while communicating to Postgres via GKE/VM.
Both Postgres (vCPU: 8 | Memory: 8GB) and GKE/VM (e2-highcpu-2are hosted under the same region and zone while they connect to each other privately under the same VPC.
When I use psql via GKE/VM and run a query (instructions below to reproduce), it seems to take 8-10 ms and apparently most of the time is taken over the network as Query Insights at Postgres shows the query execution to be taking 0.15 ms.
On my local, it takes around 1 ms or less, should I expect it to be so much higher on GCP?
Does anyone know if this is expected or there is any possible misconfiguration leading to it?
 
Here are the commands I have tried over a pod at GKE or VM:
VM:
$ sudo apt-get install -y postgresql-client
$ PGPASSWORD=user psql --username=user --host=10.7.2.3 --port=5432
user=# \timing
user=# SELECT count(*) FROM pg_tables;
count
-------
81
(1 row)
Time: 8.771 ms
Pod at GKE:
$ apk add postgresql-client
$ PGPASSWORD=user psql --username=user --host=10.7.2.3 --port=5432
user=# \timing
user=# SELECT count(*) FROM pg_tables;
count
-------
81
(1 row)
Time: 10.055 ms

 
0 1 1,028
1 REPLY 1

It shouldn't be, the normal latency in the same zone is about 80 microseconds, you can find the information about latencies in this link.

Now, with the information provided it is very hard to troubleshoot this issue, I would recommend you to create a support case through the Support Hub page, someone from GCP will follow up.