Assistance Required for Estimating Hosting and Deployment Costs for Fine-Tuning Gemini 1.0 Pro

[Adityapurohit]

Hello Google Cloud Community,

I am fine-tuning Gemini 1.0 Pro on a small dataset of approximately 5-10 MB and need your help determining the hosting, deployment, and training costs. My goal is to get an accurate estimate of the infrastructure requirements and associated costs. Although I have reviewed the Gemini token pricing and tried using the Google Cloud Pricing Calculator, I couldn’t find detailed information that aligns with my specific use case.

Details of My Use Case:

• Model Type: Gemini 1.0 Pro
• Dataset Size: ~5-10 MB
• Training Objective: Fine-tuning the model for a niche use case
• Requirement:
  1. Machine Type – Which machine type would be optimal for efficient training (e.g., GPU/TPU requirements)?
  2. Hosting Costs – Expected monthly charges for hosting the trained model for inference.
  3. Training Costs – Approximate costs for a complete training session on this dataset.

I would appreciate any insights or recommendations that can guide me toward a more accurate cost estimation. Thank you for your time and expertise!

[abhishekbhagwat]

Hi! 
We offer a managed tuning service for Gemini - https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini-use-supervised-tuning

Tuning is charged for the total number of tokens in your dataset rather than the machine costs. 
Predictions on the tuned endpoints are the same as the untuned gemini endpoints, we do not charge anything extra for inference on the tuned endpoints - https://cloud.google.com/vertex-ai/generative-ai/pricing#gemini-models

Hope this helps!

[dawnberdan · Accepted solution]

Hi @Adityapurohit, 

Welcome to Google Cloud Community!

In addition to @abhishekbhagwat, to get a better estimate of the costs for fine-tuning Gemini 1.0 Pro on your dataset, let's break down your requirements into three main areas: machine type, hosting costs, and training costs.

1. Machine Type for training: For fine-tuning models like Gemini 1.0 Pro, using a machine with GPU or TPU capabilities is ideal. Here are some suggestions:

• NVIDIA GPUs (e.g., A100, V100): These are powerful and suitable for deep learning tasks. A machine with a single A100 GPU should suffice for your dataset size.
• TPUs (e.g., v2 or v3): If you're looking for a more managed solution, TPUs can be very efficient for training, particularly if your model is compatible.

Recommendation: If your budget allows, consider an n1-standard-8 instance with a single A100 GPU or a TPU v2 for efficient training.

2. Hosting Costs: For hosting the trained model for inference, costs depend on the expected traffic and usage patterns. You might consider the following:

• Instance Type: An n1-standard-2 or n1-standard-4 instance could be appropriate for hosting, depending on the expected load.
• Monthly Costs: Depending on usage, this typically costs between $50 and $200. You may also incur additional charges for storage and network egress.

3. Training Costs: Training costs depend on the duration and type of resources used.

To get an estimate, try using the Google Cloud Pricing Calculator with your specific configurations. It’ll provide figures based on your usage. 

I hope the above information is helpful. 

[Adityapurohit]

Thankyou for this information 😊