Co-hosting Pytorch models on Vertex AI

[xolisani]

Hi, 

I'm trying to co-host pytorch models on vertex following https://cloud.google.com/blog/products/ai-machine-learning/introducing-co-hosting-models-on-the-vert... from July 2022. On the article it say support is only for TF models. Is this still the case? if so what is the ETA on Pytorch support? Also, is the a current manual workaround for Pytorch models?

[kvandres · Accepted solution]

Good day @xolisani,

Welcome to Google Cloud Community!

As of now, it only supports Tensorflow models, but please note that a feature request was filed to include pytorch models and the request was already forwarded to the Vertex AI Engineering Team for evaluation although currently there is no exact ETA when it will be available but you can track the progress using this link: https://issuetracker.google.com/255271990
As a workaround, you can try packing all your models in one container and use a custom HTTP server logic  for prediction requests that uses the parameters field in the prediction request body in order to send it to the correct model. You can check this link that I found for more information, please note that this is not supported by Google but I am sending it to you since it might be helpful to your case: https://stackoverflow.com/questions/69878915/deploying-multiple-models-to-same-endpoint-in-vertex-ai

Hope this is useful!