I am trying to connect to a Cloud SQL database during a Cloud Build step so I can run database migrations for my app. But I keep running into errors such as:
SQLSTATE[HY000] [2002] Operation timed out
I have followed the instructions here very closely:
https://cloud.google.com/sql/docs/mysql/connect-build#php_2
Here is my cloudbuild.yaml:
Hi @vincemcgaj,
Welcome to the Google Cloud Community!
You can try the following troubleshooting options:
In Configure Cloud Build, under the Private IP tab, Cloud Build must be in the same VPC network as your Cloud SQL Instance. To configure this:
You should also take a look at this Stack Overflow Post as you might have the same problem. Also, according to this post, Cloud Build does not support VPC networks, therefore it is not possible to connect to the private IP of the Cloud instance. Which contradicts the above option.
I suggest contacting Google Cloud Support to further look into your case. Let me know if it helped, thanks!
@Marramirez Thank you for the reply. It would make sense if you cannot connect to a Cloud SQL database from Cloud Build through a private IP because I am fairly certain I have done everything right and followed all those steps listed and I am able to connect through other service such as Cloud Run or BigQuery. However, the documentation provided by Google suggests otherwise because you have a Public IP option and a Private IP options. I'm not fully satisfied by that stack overflow post because as you said the documentation contradicts this and also why would there be Cloud Build Worker Pools that connect to a VPC if Cloud Build does not support VPC? That is confusing.
I noticed with BigQuery that I was able to make a private connection by just providing the connection name and no IP address. Should I be connecting through the DB_SOCKET? I thought socket connections did not work for private connections? Whenever I tried to connect a Cloud function or a Cloud Run service to Cloud SQL it requires the private IP. But BigQuery required the connection name instead. I have tried connecting through DB_SOCKET as well before, no luck.
Hi @vincemcgaj , were you able to get this issue resolved. I am having a similar issue connecting to cloud SQL from Cloud build.
@milindajuan unfortunately, no I have not found a solution. I just gave up and switched to using a public connection which works just fine. That's what I suggest. Public connection is still pretty secure and you do not need to add an IP address to the Cloud SQL connection list for the Cloud Build. Just leave the connections list on your Cloud SQL connections empty and no one can publicly connect.
Good luck if you are going to try this. I have already wasted a lot of time trying and I don't think you can actually connect to Cloud SQL from Cloud Build through VPC connection. So my warning is don't waste time on this and just use a public connection.
Hey @vincemcgaj , I was able to resolve the issue with the cloud build workerPool. Once we create the worker pool and run the build in there, it connects without an issue. Thank you for your respopnse.
@milindajuan Really? I did that as well, connected using a worker pool and it never worked. Either I'm doing something wrong or Google fixed something to make this possible. I'm probably just doing something wrong, oh well. Feel free to share your code, it may become very helpful to someone in the future.
Basically this is my test image - for anyone with future need.
steps:
# build the container image
- id: 'build'
name: 'docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/image-x', '.']
- id: 'push'
name: 'docker'
args: ['push', 'gcr.io/$PROJECT_ID/image-x']
# Running Tests
- id: 'test'
name: 'gcr.io/$PROJECT_ID/image-x'
entrypoint: 'bash'
env:
- 'DATABASE_URL_TEST=sqlite:///:memory:'
secretEnv:
- DATABASE_URL
args:
- '-c'
- |
chmod +x bin/console
composer test:build
composer build:database
# Deploy container image to Cloud Run
- id: 'deploy'
name: 'gcr.io/cloud-builders/gcloud'
args: ['beta', 'run', 'deploy', 'image-x', '--image', 'gcr.io/$PROJECT_ID/image-x', '--region', 'australia-southeast1', '--platform', 'managed','--allow-unauthenticated']
options:
pool:
name: projects/$PROJECT_ID/locations/australia-southeast1/workerPools/sql-connector
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/DATABASE_URL/versions/latest
env: DATABASE_URL
Thank you for the reply. It looks like you are connecting to a sqlite database for unit test purposes. I'm not trying to do that, I'm trying to connect to a Cloud SQL database so we can run database migrations on the production MySQL database when deploying newer versions of our app. I do not think this will work for me. But thank you for trying.
Nope. You understood incorrectly. If you look closer there are 2 variables. One for the test and the other for the actual database. Sqllite is to run integration tests and the actual connection DATABASE_URL is to run the migrations in the cloud SQL.
@milindajuan Interesting. I'm assuming you are running the migration in
composer build:database
I've never seen it done like this. I might try this myself but there are no commands named build in my namespace. So would we have to define this method ourselves?