Unable to Push to Bitbucket from Dataform via SSH

[s-yuuki]

Hello,

I am currently using Google Cloud Dataform to manage a Bitbucket repository via SSH.

While SSH authentication succeeds (ssh -T git@bitbucket.org works fine), I encounter an error when trying to push changes from Dataform.

Issue

When attempting to push, I receive the following error:

Remote repository REPOSITORY_NAME could not be reached.

Expected Behavior

Dataform should successfully push changes to the Bitbucket repository via SSH.

Environment & What I’ve Tried:

•Cloud Service: Google Cloud Dataform

• Git Repository: Bitbucket (git@bitbucket.org:[REPOSITORY_NAME].git)

•SSH Key Type: Ed25519 (id_ed25519)

•Secret Manager: Private key stored and configured

•Firewall: Fully open (0.0.0.0/0) for testing

•Bitbucket Repository: Set to Public temporarily for troubleshooting

I have already verified that:

✅ SSH authentication works (ssh -T git@bitbucket.org returns authenticated via ssh key.).

✅The SSH public key is registered with Bitbucket.

✅The private key is stored in Google Cloud Secret Manager and configured in Dataform.

✅ The firewall allows all outgoing traffic for testing.

✅ The repository has been made public to rule out permission issues.

Despite these checks, Dataform cannot push changes.

My Questions

•Are there additional configurations needed for Dataform to push changes via SSH?

•How can I verify that Dataform is correctly using the SSH key stored in Secret Manager?

•Are there any known restrictions on Dataform’s SSH push functionality?

• Should I check protoPayload.serviceName="dataform.googleapis.com" logs for more details?

I would appreciate any insights on how to resolve this issue.

If additional logs or configurations are needed, please let me know!

Thank you!

[NorieRam]

Hi @s-yuuki,

Welcome to Google Cloud Community!

Here's a breakdown to your questions:

• Are there additional configurations needed for Dataform to push changes via SSH?

For configuration, you can refer to Connect a remote repository through SSH. Ensure the "Git remote" in your Dataform project settings exactly matches your SSH URL. Also, double-check the service account used by Dataform has "Secret Manager Secret Accessor" permissions.

• How can I verify that Dataform is correctly using the SSH key stored in Secret Manager?

Dataform doesn't offer direct key verification, checking Dataform logs (specifically for dataform.googleapis.com) is an approach to debug connection issues and look for SSH-related errors.

• Are there any known restrictions on Dataform’s SSH push functionality?

Dataform's SSH functionality is generally robust, but there might be transient network issues. Consider testing with a different SSH key type as a diagnostic step.

• Should I check protoPayload.serviceName="dataform.googleapis.com" logs for more details?

Yes, the protoPayload.serviceName="dataform.googleapis.com" logs are crucial for troubleshooting. You can filter for errors related to your repository and SSH connection attempts.

Check this related discussion, as it might help resolve your situation as well.

If the issue persists, please contact Google Cloud Support. When reaching out, include detailed information and relevant screenshots of the errors you’ve encountered. This will assist them in diagnosing and resolving your issue more efficiently.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[idejuan]

hi @s-yuuki 

I am experiencing a similar problem when using BigQuery Repositories - a new feature in BigQuery that uses Dataform git integration.

The issue I am experiencing only applies to PUSH but not to PULL. PULL is working correctly, so I can pull the changes from Bitbucket into Bigquery Repositories. This proves that access to the external Bitbucket repo is working well, and it is only an issue of lack of permissions to write in the Bitbucket repo.

But for PULL I get the same error response as you do:

Remote repository REPOSITORY_NAME could not be reached.

May I ask if you have the problem only for PUSH but not for PULL as I do?

Thank you!