Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Vector Search Quickstart Error

I am following the Vector Search Quickstart Guide, https://cloud.google.com/vertex-ai/docs/vector-search/quickstart#install-sdk, and encountering an issue when trying to run the following query in Jupyter notebook: 

# run query
response = my_index_endpoint.find_neighbors(
deployed_index_id = DEPLOYED_INDEX_ID,
queries = [query_emb],
num_neighbors = 10
)

# show the results
for idx, neighbor in enumerate(response[0]):
print(f"{neighbor.distance:.2f} {product_names[neighbor.id]}")

However, every time I execute this, I receive the following error:

---------------------------------------------------------------------------
_InactiveRpcError                         Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:65, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     64 try:
---> 65     return callable_(*args, **kwargs)
     66 except grpc.RpcError as exc:

File /opt/conda/lib/python3.10/site-packages/grpc/_interceptor.py:277, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
    268 def __call__(
    269     self,
    270     request: Any,
   (...)
    275     compression: Optional[grpc.Compression] = None,
    276 ) -> Any:
--> 277     response, ignored_call = self._with_call(
    278         request,
    279         timeout=timeout,
    280         metadata=metadata,
    281         credentials=credentials,
    282         wait_for_ready=wait_for_ready,
    283         compression=compression,
    284     )
    285     return response

File /opt/conda/lib/python3.10/site-packages/grpc/_interceptor.py:332, in _UnaryUnaryMultiCallable._with_call(self, request, timeout, metadata, credentials, wait_for_ready, compression)
    329 call = self._interceptor.intercept_unary_unary(
    330     continuation, client_call_details, request
    331 )
--> 332 return call.result(), call

File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:440, in _InactiveRpcError.result(self, timeout)
    439 """See grpc.Future.result."""
--> 440 raise self

File /opt/conda/lib/python3.10/site-packages/grpc/_interceptor.py:315, in _UnaryUnaryMultiCallable._with_call.<locals>.continuation(new_details, request)
    314 try:
--> 315     response, call = self._thunk(new_method).with_call(
    316         request,
    317         timeout=new_timeout,
    318         metadata=new_metadata,
    319         credentials=new_credentials,
    320         wait_for_ready=new_wait_for_ready,
    321         compression=new_compression,
    322     )
    323     return _UnaryOutcome(response, call)

File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1198, in _UnaryUnaryMultiCallable.with_call(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1192 (
   1193     state,
   1194     call,
   1195 ) = self._blocking(
   1196     request, timeout, metadata, credentials, wait_for_ready, compression
   1197 )
-> 1198 return _end_unary_response_blocking(state, call, True, None)

File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1006, in _end_unary_response_blocking(state, call, with_call, deadline)
   1005 else:
-> 1006     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:34.36.100.88:443: Failed to connect to remote host: Timeout occurred: FD Shutdown"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-03-26T15:00:40.03179638+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:34.36.100.88:443: Failed to connect to remote host: Timeout occurred: FD Shutdown"}"
>

The above exception was the direct cause of the following exception:

ServiceUnavailable                        Traceback (most recent call last)
Cell In[14], line 2
      1 # run query
----> 2 response = my_index_endpoint.find_neighbors(
      3     deployed_index_id = DEPLOYED_INDEX_ID,
      4     queries = [query_emb],
      5     num_neighbors = 10
      6 )
      8 # show the results
      9 for idx, neighbor in enumerate(response[0]):

File /opt/conda/lib/python3.10/site-packages/google/cloud/aiplatform/matching_engine/matching_engine_index_endpoint.py:1700, in MatchingEngineIndexEndpoint.find_neighbors(self, deployed_index_id, queries, num_neighbors, filter, per_crowding_attribute_neighbor_count, approx_num_neighbors, fraction_leaf_nodes_to_search_override, return_full_datapoint, numeric_filter, embedding_ids, signed_jwt, psc_network)
   1697     find_neighbors_query.datapoint = datapoint
   1698     find_neighbors_request.queries.append(find_neighbors_query)
-> 1700 response = self._public_match_client.find_neighbors(find_neighbors_request)
   1702 # Wrap the results in MatchNeighbor objects and return
   1703 return [
   1704     [
   1705         MatchNeighbor(
   (...)
   1714     for embedding_neighbors in response.nearest_neighbors
   1715 ]

File /opt/conda/lib/python3.10/site-packages/google/cloud/aiplatform_v1beta1/services/match_service/client.py:799, in MatchServiceClient.find_neighbors(self, request, retry, timeout, metadata)
    796 self._validate_universe_domain()
    798 # Send the request.
--> 799 response = rpc(
    800     request,
    801     retry=retry,
    802     timeout=timeout,
    803     metadata=metadata,
    804 )
    806 # Done; return the response.
    807 return response

File /opt/conda/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py:113, in _GapicCallable.__call__(self, timeout, retry, *args, **kwargs)
    110     metadata.extend(self._metadata)
    111     kwargs["metadata"] = metadata
--> 113 return wrapped_func(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:67, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
     65     return callable_(*args, **kwargs)
     66 except grpc.RpcError as exc:
---> 67     raise exceptions.from_grpc_error(exc) from exc

ServiceUnavailable: 503 failed to connect to all addresses; last error: UNKNOWN: ipv4:34.36.100.88:443: Failed to connect to remote host: Timeout occurred: FD Shutdown


I am not sure why this is.

0 2 577
2 REPLIES 2

Hi @JackJonesTrellx,

Welcome to the Google Cloud Community!

The error message "ServiceUnavailable: 503 failed to connect to all addresses; last error: UNKNOWN: ipv4:34.36.100.88:443: Failed to connect to remote host: Timeout occurred: FD Shutdown" signifies that your client is experiencing connectivity issues with the Vertex AI Matching Engine service.

Here are the potential ways that might help with your use case:

  • Firewall Rule Check: Verify that your firewall rules permit outbound traffic on port 443, and ensure your network configuration is set up to allow outbound traffic to Google Cloud services.
  • Region Consistency: Make sure that your notebook and the Vertex AI resources (such as the index and index endpoint) are in the same region. Connectivity issues can arise from region mismatches. Check the region specified in your code as well as in the Vertex AI console.
  • Service Account Permissions: Ensure that your service account is granted both the "roles/aiplatform.indexUser" and "roles/aiplatform.viewer" roles. Without the appropriate permissions, your code will be unable to access Vertex AI Matching Engine, resulting in errors and rendering your vector search non-functional, regardless of network connectivity.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

Thank you for your help however there are no Fireqall Rules, the Regions are consistent, and the service account has the needed permissions for aiplatform.