LinkedIn
Twitter
My Vertex AI instance will not start after an auto-upgrade that was preformed on it. It seems to forever be stuck on "Starting", "Initializing" or "Provisioning". Using the diagnostic tool it appears the Jupyter is "unhealthy":
Using the Google Cloud console to attempt to restart the service is not helpful as I get the following error:
ERROR: (gcloud.compute.start-iap-tunnel) Error while connecting [4003: 'failed to connect to backend']. (Failed to connect to port 22)
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
However, I have setup the firewall permissions to allow such connections as outlined in the documentation.
Running the subsequent troubleshooter that is recommended to resolve the issue results in the following error:
Starting ssh troubleshooting for instance https://compute.googleapis.com/compute/v1/projects/future-pager-399113/zones/us-east4-a/instances/PR... in zone us-east4-a
Start time: 2024-04-17 14:38:16.717230
---- Checking network connectivity ----
The Network Management API is needed to check the VM's network connectivity.
Is it OK to enable it and check the VM's network connectivity? (Y/n)? Y
Enabling service [networkmanagement.googleapis.com] on project [future-pager-399113]...
Your source IP address is 35.231.107.27
Network Connectivity Test Result: UNREACHABLE
To view complete details of this test, see https://console.cloud.google.com/net-intelligence/connectivity/tests/details/ssh-troubleshoot-v6a48?...
Help for connectivity tests:
https://cloud.google.com/network-intelligence-center/docs/connectivity-tests/concepts/overview
---- Checking user permissions ----
User permissions: 0 issue(s) found.
---- Checking VPC settings ----
VPC settings: 0 issue(s) found.
---- Checking VM status ----
The Monitoring API is needed to check the VM's Status.
Is it OK to enable it and check the VM's Status? (Y/n)? Y
Enabling service [monitoring.googleapis.com] on project [future-pager-399113]...
ERROR: (gcloud.compute.ssh) HTTPError 400: The resource 'projects/future-pager-399113/zones/us-east4-a/instances/PROJECT' is not ready
This all seems to be circular and initially was caused by Googles auto-upgrade service. What are the steps to resolve this?
