Currently, team is connecting from on prem server to cloudsql using public IP.
they have whitelisted server IP in authorized networks.
What is the more secure way to connect here? thinking of CloudSQL Auth proxy here, if they install CloudSQL Auth proxy on that on prem server and run it as service , is it enough or any other more secure way to connect?
Yep, using the Cloud SQL Auth Proxy is a more secure way to connect. However, note that the service account used by the proxy should be configured with the least privilege, ideally - with a custom role that only grants access to a specific Cloud SQL instance.