vertex-AI-model-deployment

[shibom93]

Hi, is there any option to deploy existing model in vertex-AI such as Palm2 for text. or if there is any option to get an API key for Palm API. as I have raised a request for Palm API key, however still not received yet.

[cincyohio]

Hopefully someone can answer this. Also curious, as the OP.

[shakeebs]

There isn't an API key that you can specifically generate for palm2 for text. If you go to Model Garden, and click the view details on the model you would like to run, it will have instructions on how to set up palm2 API. 

Here are the instructions I've copied from there:
1. run

gcloud auth application-default login

 2. replace project_id with your own GCP project_id and do this `POST` request

curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
  "instances": [
    { "prompt": "Give me ten interview questions for the role of program manager."}
  ],
  "parameters": {
    "temperature": 0.2,
    "maxOutputTokens": 256,
    "topK": 40,
    "topP": 0.95
  }
}'

[shibom93]

Can I deploy the same Palm2 model for text in vertex AI model deployment to use it for own R&D?

[shakeebs]

If you're asking about deploying palm2 in your own instance, palm2 is not an open source model and thus you won't be able to run it in your own instance like you can with llama2. 

You can connect to it via the PaLM API or using language specific SDKs

[Poala_Tenorio]

Google Vertex AI did not provide a pre-built service for deploying models like Palm2 for text specifically. However, Google Cloud does offer various tools and services for deploying custom machine learning models. You can use Vertex AI or other Google Cloud services like AI Platform to deploy your own custom text models.